Airflow

Introducing Airflow

Airflow release comes with exciting improvements for installing libraries.

• Use pip only after using Conda.

  • Install as many requirements as possible with Conda and then use pip.

  • Pip should be run with –upgrade-strategy only-if-needed (the default).

  • Do not use pip with the –user argument; avoid all “users” installations.

• Use Conda environments for isolation.

  • Create a Conda environment to isolate any changes pip makes.

  • Hard links cause environments to take up little space.

  • Avoid running pip in the “root” environment.

• Recreate the environment if changes are needed.

  • Once pip has been used, Conda will be unaware of any changes.

  • Recreate the environment to install additional Conda packages.

• Store Conda and pip requirements in text files.

  • Package requirements can be passed to Conda via the –file argument.

  • Pip accepts a list of Python packages with -r or –requirements.

  • Conda env will export or create environments based on a file with both, Conda and pip requirements.