Overview

In this tutorial, you will learn how to set environment variables in Ubuntu, CentOS, Red Hat, basically any Linux distribution for a single user and globally for all users. You will also learn how to list all environment variables and how to unset (clear) existing environment variables.

A utility tool to create.env files ¶ dump-env takes an.env.template file and some optional environmental variables to create a new.env file from these two sources. No external dependencies are used. And while this file can be edited directory, it is actually recommended to store global environment variables in a directory named /etc/profile.d, where you will find a list of files that are used to set environment variables for the entire system. Create a new file under /etc/profile.d to store the global environment variable(s). Create a new environment ENVNAME with Python version 3.X: conda create -name ENVNAME python=3.X: Create a new environment ENVNAME with some initial packages: conda create -name ENVNAME python=3.X pandas ipykernel: Create a new environment from a yaml file: conda env create -file environment.yaml: Activate the environment ENVNAME (OSX.

Environment variables are commonly used within the Bash shell. It is also a common means of configuring services and handling web application secrets.

It is not uncommon for environment specific information, such as endpoints and passwords, for example, to be stored as environment variables on a server. They are also used to set the important directory locations for many popular packages, such as JAVA_HOME for Java.

Setting an Environment Variable

To set an environment variable the export command is used. We give the variable a name, which is what is used to access it in shell scripts and configurations and then a value to hold whatever data is needed in the variable.

For example, to set the environment variable for the home directory of a manual OpenJDK 11 installation, we would use something similar to the following.

To output the value of the environment variable from the shell, we use the echo command and prepend the variable’s name with a dollar ($) sign.

And so long as the variable has a value it will be echoed out. If no value is set then an empty line will be displayed instead.

Unsetting an Environment Variable

To unset an environment variable, which removes its existence all together, we use the unset command. Simply replace the environment variable with an empty string will not remove it, and in most cases will likely cause problems with scripts or application expecting a valid value.

To following syntax is used to unset an environment variable

For example, to unset the JAVA_HOME environment variable, we would use the following command.

Listing All Set Environment Variables

To list all environment variables, we simply use the set command without any arguments.

An example of the output would look something similar to the following, which has been truncated for brevity.

Persisting Environment Variables for a User

When an environment variable is set from the shell using the export command, its existence ends when the user’s sessions ends. This is problematic when we need the variable to persist across sessions.

To make an environment persistent for a user’s environment, we export the variable from the user’s profile script.

1. Open the current user’s profile into a text editor
   
2. Add the export command for every environment variable you want to persist.
   
3. Save your changes.

Adding the environment variable to a user’s bash profile alone will not export it automatically. However, the variable will be exported the next time the user logs in.

To immediately apply all changes to bash_profile, use the source command.

Export Environment Variable

Export is a built-in shell command for Bash that is used to export an environment variable to allow new child processes to inherit it.

How To Create An Env File

To export a environment variable you run the export command while setting the variable.

We can view a complete list of exported environment variables by running the export command without any arguments.

To view all exported variables in the current shell you use the -p flag with export.

Setting Permanent Global Environment Variables for All Users

A permanent environment variable that persists after a reboot can be created by adding it to the default profile. This profile is loaded by all users on the system, including service accounts.

All global profile settings are stored under /etc/profile. And while this file can be edited directory, it is actually recommended to store global environment variables in a directory named /etc/profile.d, where you will find a list of files that are used to set environment variables for the entire system.

1. Create a new file under /etc/profile.d to store the global environment variable(s). The name of the should be contextual so others may understand its purpose. For demonstrations, we will create a permanent environment variable for HTTP_PROXY.
   
2. Open the default profile into a text editor.
   
3. Add new lines to export the environment variables
   
4. Save your changes and exit the text editor

Conclusion

This tutorial covered how to set and unset environment variables for all Linux distributions, from Debian to Red Hat. You also learned how to set environment variables for a single user, as well as all users.

Save the environment with conda (and how to let others run your programs)

If you have been developing in Python, you may have tried to distribute your program to friends and colleagues. It can be mildly annoying when they try to run your program and it fails because they don't have obscurePackage42 installed. If you are nearby, then it is easy for you to call pip install a few times and get them started with your program. If you are trying to distribute a program to end users (or even some non-technical executives) then you really want something that is going to work 'out of the box'.

Using an environment has the additional benefit of having us deal with one specific known version of Python. The problem of 'which Python am I using?' is one familiar to many of us.

The old way (and its drawbacks)

One way of doing this was to write a requirements.txt file. The format of this file was pretty simple:

A single command, pip install -r requirements.txt and everything would be written to the main Python repository.

While simple to use, there are a couple of different problems with this approach:

• Version conflicts: What if one application required version 0.23.4 of Pandas, but a different application required 0.19.0 (because it used a now deprecated feature)? We would have to reinstall from requirements.txt when switching between these applications.
• Tracking dependencies: It can be difficult to keep track of which packages your application is actually using. You don't want to include all installed packages on your machine, as only a few are relevant to your application.

Environments were designed to address both of these issues.

Environments

An environment is a way of starting with a new Python installation, that doesn't look at your already installed packages. In this way, it simulates having a fresh install of Python. If two applications require different versions of Python, you can simply create different environments for them. If you start from a fresh environment and install as you go, you are able to generate a list of all packages installed in the environment so that others can easily duplicate it.

There are many different environments and dependency managers in the Python ecosystem. The most common ones in use are virtualenv and conda (but there are others such as poetry, pyenv/pipenv, hatch and many more I haven't heard of). This article is about using conda to manage environments, although all of these tools share the same broad goals. Some of the differences between these tools are touched on in the Alternatives section.

There are two steps to using an environment (with a third step needed if you want to use Jupyter notebooks)

1. Creating the environment, either from scratch (a new project) or from a yaml file (duplicating an environment)
2. Activating the environment for use.
3. Register the environment with Jupyter.

To leave an environment, we have to deactivate it. The quickstart below will walk through the typical workflow.

Using an environment (quickstart)

Let's say you wanted to create an environment test_env to do some testing with Python 3.6, and install numpy and Pandas. At the terminal, type the following:

If you want Jupyter notebooks to see your new environment, you need a couple of extra instructions. Jupyter sees the different environments as different kernels. Once we create a new environment, we need to tell Jupyter that it is there:

When loading a jupyter notebook, you can use the menu options Kernel->Change Kernel->test kernel to ensure you are using the test_env environment. (Update 2019-11-13: This process doesn't always work, this article gives you ways of checking and fixing it if it doesn't.)

Now you want to make an environment.yaml file that will allow others to recreate the environment from scratch. To make this file, we use the export command and send the output to environment.yaml:

Once we are done with the environment, we can deactivate and delete the environment:

Making the environment again from the yaml file

If you have the yaml file (created from conda env export), then recreating the environment is a single command:

Note that you don't need to supply the name of the new environment, as the yaml file also contains the name of the environment it saved. Make sure you don't give your environment an embarassing name, as everyone who recreates from the yaml file will see the name you used!

Finding conda environments on your system

Of course, you may choose to deactivate your environment but keep it around for later. If you want to see the environments installed on your system, use

Useful commands

Here is a brief summary of useful commands for environemnts. Anaconda has also published a cheat sheet with a more extensive list of useful commands.

Command
Create a new environment ENV_NAME with Python version 3.X	conda create --name ENV_NAME python=3.X
Create a new environment ENV_NAME with some initial packages	conda create --name ENV_NAME python=3.X pandas ipykernel
Create a new environment from a yaml file	conda env create --file environment.yaml
Activate the environment ENV_NAME (OSX, Linux)	source activate ENV_NAME
Activate the environment ENV_NAME (Windows)	activate ENV_NAME
Deactivate the current environment (*)	source deactivate
Delete the environment ENV_NAME	conda env remove --name ENV_NAME
List all installed environments	conda env list
Create a YAML file for active environment(*)	conda env export > environment.yaml

The commands with (*) require you to have the environment active before using them. The naming is a little odd for creating environments: if creating them yourself the command is conda create ....., but if creating them from a yaml file we use conda env create ....... This is not a typo!

Tip for maximizing portability

Create Env File Nodejs

Some packages are system dependent (e.g. the Python Image Library Pillow is used by OSX and Linux, but not Windows). Once you create your environment.yaml file, it is often a good idea to eliminate packages you don't use directly. For example, if you tell conda to install pandas but not numpy it will figure out it needs numpy for pandas to work. You should aim to eliminate all the packages in environment.yaml except the ones you actually import, so that conda can figure out which other packages are needed for the user's system (which may be running a different operating system to yours).

Summary

Creating environments allow us to make sure users we distribute our code to have the right packages (and the right versions of those packages installed) to run our code, without interfering with other programs. We activate and environment to start using it, and deactivate to leave again.

If making a new environment that you want others to use, the workflow is

If you are using an environment someone else has created:

Alternatives

Create .env File Windows

• The original virtualenv. As the Jake VanderPlas article 'Conda: Myths and Misconceptions' points out, these are mostly interchangle if you are only installing python packages into your environment. In slightly more detail
  • virtualenv/pip installs python packages into any environment, while
  • conda installs any packages into conda environments.

If you are solely installing Python packages, there is not much difference between the two.

Activate Conda Environment In Powershell

• pyenv/pipenv by Kenneth Reitz. The main goal of this project was to automate/simplify environment creation, but is not as mature as either virtualenv or conda solutions.
• poetry by Sébastien Eustace, which aims to be a packaging and deployment tool.

With the exception of virtualenv, none of these solutions are as mature as conda. This is a piece of the Python that will hopefully improve and simplify, but for now, Randall Munroe's XKCD comic puts it well.