12. Can I use Python 2.7 for Hadoop tasks?

  • Python 2.7 is the version installed by default on cluster nodes in the following QDS accounts:

    • AWS accounts created on or after September 29 2016.
    • All Azure accounts
    • All Oracle OCI accounts
  • Python 2.6 is the default on other accounts.

12.1. Can I use Python 2.7 on an AWS account created before September 29 2016?

Yes. To enable Python 2.7 as the default version on one of these older AWS accounts, create a ticket with Qubole Support . Once Python 2.7 is enabled at the account level, Python 2.7 will be used for all clusters in this account. The Python 2.7 virtual environment no longer needs to be passed through the node bootstrap file. (Qubole has installed the Python libraries that reside in the virtual environment in the system-level Python 2.7 for a smooth migration).

12.2. Can I enable Python 2.7 for an individual cluster?

Yes. If your AWS account is using the older default Python version, 2.6, you can enable Python 2.7 for a specific cluster by adding the following lines to the node bootstrap file specified in the cluster configuration.

source /usr/lib/hustler/bin/qubole-bash-lib.sh
make-python2.7-system-default

The next invocation of the cluster will use Python 2.7.