ChatGPT解决这个技术问题 Extra ChatGPT

Set environment variables from file of key/value pairs

TL;DR: How do I export a set of key/value pairs from a text file into the shell environment?

For the record, below is the original version of the question, with examples.

I'm writing a script in bash which parses files with 3 variables in a certain folder, this is one of them:

MINIENTREGA_FECHALIMITE="2011-03-31"
MINIENTREGA_FICHEROS="informe.txt programa.c"
MINIENTREGA_DESTINO="./destino/entrega-prac1"

This file is stored in ./conf/prac1

My script minientrega.sh then parses the file using this code:

cat ./conf/$1 | while read line; do
    export $line
done

But when I execute minientrega.sh prac1 in the command line it doesn't set the environment variables

I also tried using source ./conf/$1 but the same problem still applies

Maybe there is some other way to do this, I just need to use the environment variables of the file I pass as the argument of my script.

This is a great question but is phrased way too specifically, with particular variable names ("MINIENTREGA_FECHALIMITE"? what does that mean?) and numbers (3). The general question is simply, "How do I export a set of key/value pairs from a text file into the shell environment".
Also, this has already been answered on unix.SE and is arguably more on-topic there.
A tip probably useful for beginners: Make sure you'll "execute" the script when sourcing environment variables in it. That way, you won't let them enter and pollute your own environment and ALSO otherwise can even be unsecure sometimes, for example, when you have secrets stored in one of those environment variables.

N
Nate

This might be helpful:

export $(cat .env | xargs) && rails c

Reason why I use this is if I want to test .env stuff in my rails console.

gabrielf came up with a good way to keep the variables local. This solves the potential problem when going from project to project.

env $(cat .env | xargs) rails

I've tested this with bash 3.2.51(1)-release

Update:

To ignore lines that start with #, use this (thanks to Pete's comment):

export $(grep -v '^#' .env | xargs)

And if you want to unset all of the variables defined in the file, use this:

unset $(grep -v '^#' .env | sed -E 's/(.*)=.*/\1/' | xargs)

Update:

To also handle values with spaces, use:

export $(grep -v '^#' .env | xargs -d '\n')

on GNU systems -- or:

export $(grep -v '^#' .env | xargs -0)

on BSD systems.

From this answer you can auto-detect the OS with this:

export-env.sh

#!/bin/sh

## Usage:
##   . ./export-env.sh ; $COMMAND
##   . ./export-env.sh ; echo ${MINIENTREGA_FECHALIMITE}

unamestr=$(uname)
if [ "$unamestr" = 'Linux' ]; then

  export $(grep -v '^#' .env | xargs -d '\n')

elif [ "$unamestr" = 'FreeBSD' ]; then

  export $(grep -v '^#' .env | xargs -0)

fi


Thanks, I like that this doesn't require prepending anything to the file — allows for compatibility with Foreman (Procfile) .env format.
@BenjaminWheeler GNU linux has -d for setting the delimiter, so I'm trying env $(cat .env | xargs -d '\n') rails, but it still errors with a file not found if .env has spaces. Any idea why this doesn't work?
Here's a shorter variation eval $(cat .env) rails
As a short explanation (helped me understand what the solution was actually doing), cat .env will read the .env file, then we pipe the result to xargs which is a helper tool to construct cli arguments. Finally we pass both the arguments and the command to be run to env which loads the environment variables and executes the command.
I've had problems with this solution when wrapping the variables in quotes, such as REACT_APP_IFRAME_URI="http://localhost:3000". The quotes were parsed which wasn't intended.
f
fedorqui

-o allexport enables all following variable definitions to be exported. +o allexport disables this feature.

set -o allexport
source conf-file
set +o allexport

Works like a charm! Even if .env file has comments in it. Thanks!
And in one line set -o allexport; source conf-file; set +o allexport
This is a great way to read in a properties file, when the Jenkins EnvInject plug-in doesn't work. Thanks!
@CMCDragonkai, for POSIX, it would be set -a; . conf-file; set +a.
This method works if the environment variables has spaces in it. Many of the others do not. While the eval() method does, I also get a little weirded out by using eval
b
b4hand

Problem with your approach is the export in the while loop is happening in a sub shell, and those variable will not be available in current shell (parent shell of while loop).

Add export command in the file itself:

export MINIENTREGA_FECHALIMITE="2011-03-31"
export MINIENTREGA_FICHEROS="informe.txt programa.c"
export MINIENTREGA_DESTINO="./destino/entrega-prac1"

Then you need to source in the file in current shell using:

. ./conf/prac1

OR

source ./conf/prac1

Although reading the file line-by-line and passing each line to export is not ideal, the problem can also be fixed by simply using input redirection on the loop: while read line; do ... ; done < ./conf/$1.
And if it's not from a file, use < <(commands that generate output)
You have a more clean solution, I have a preference for set -o allexport
If using this .env file between systems, inserting export would break it for things like Java, SystemD, or other tools
awk '{print "export " $0}' envfile convenience command to prepend export to the beginning of every line
c
cglacet
set -a
. ./env.txt
set +a

If env.txt is like:

VAR1=1
VAR2=2
VAR3=3
...

Explanations -a is equivalent to allexport. In other words, every variable assignment in the shell is exported into the environment (to be used by multiple child processes). More information can be found in the Set builtin documentation:

-a Each variable or function that is created or modified is given the export attribute and marked for export to the environment of subsequent commands. Using ‘+’ rather than ‘-’ causes these options to be turned off. The options can also be used upon invocation of the shell. The current set of options may be found in $-.


what if there is a comment? and VAR2=$VAR1?
Hi @Alexis. The . command used here is essentially like typing on the terminal. You can try it yourself on the terminal and see what the results would be. Comments will be ignored and references to other variables will work so long as they have been defined earlier.
Yes, I tried it was working that way. Thanks for the follow-up!
H
Huan

I found the most efficient way is:

export $(xargs < .env)

Explanation

When we have a .env file like this:

key=val
foo=bar

run xargs < .env will get key=val foo=bar

so we will get an export key=val foo=bar and it's exactly what we need!

Limitation

It doesn't handle cases where the values have spaces in them. Commands such as env produce this format. – @Shardj


Doesn't work on MacOS :(
Thanks! It works on my Monterey 12.4 macOS
g
gsf

The allexport option is mentioned in a couple of other answers here, for which set -a is the shortcut. Sourcing the .env really is better than looping over lines and exporting because it allows for comments, blank lines, and even environment variables generated by commands. My .bashrc includes the following:

# .env loading in the shell
dotenv () {
  set -a
  [ -f .env ] && . .env
  set +a
}

# Run dotenv on login
dotenv

# Run dotenv on every new directory
cd () {
  builtin cd $@
  dotenv
}

This looks nice, but you do you unload environment variables when you leave the directory?
I don't unset variables, and it's never been a problem. My apps tend to use variable names that are distinct, and if there is overlap, I'll set them to blank in that .env with VAR=.
A
Amit Joki
eval $(cat .env | sed 's/^/export /')

Using eval $(cat .env | sed 's/^[^$]/export /') allows you to have empty lines for better readability.
I find that cat .env | sed 's/^[^$]/export /' strips off the initial character. I.e. for a file A=foo\nB=bar\n I get export =foo\nexport =bar\n. This works better for skipping blank lines: cat .env | sed '/^$/! s/^/export /'.
(I also note for the sake of UNIX code golfers that you don't need cat in either case: eval $(sed 's/^/export /' .env) works just as well.)
dont'support commented row initial with #
eval sed 's/^/export /' .env
k
kolypto

The problem with source is that it requires the file to have a proper bash syntax, and some special characters will ruin it: =, ", ', <, >, and others. So in some cases you can just

source development.env

and it will work.

This version, however, withstands every special character in values:

set -a
source <(cat development.env | \
    sed -e '/^#/d;/^\s*$/d' -e "s/'/'\\\''/g" -e "s/=\(.*\)/='\1'/g")
set +a

Explanation:

-a means that every bash variable would become an environment variable

/^#/d removes comments (strings that start with #)

/^\s*$/d removes empty strings, including whitespace

"s/'/'\\\''/g" replaces every single quote with '\'', which is a trick sequence in bash to produce a quote :)

"s/=\(.*\)/='\1'/g" converts every a=b into a='b'

As a result, you are able to use special characters :)

To debug this code, replace source with cat and you'll see what this command produces.


Works for me (TM) on bash, using the following annoying string: FOO=~`#$&*()\|[=]{}; '"<>/?!
Great, may thanks for this!
This almost worked for me, but had to swap out \s with [[:space:]] to make it also work on bash on FreeBSD/Mac: ``` source <(cat .env | sed -e '/^#/d;/^[[:space:]]*$/d' -e "s/'/'\\\''/g" -e "s/=(.*)/='\1'/g") ```
t
tutuDajuju

Here is another sed solution, which does not run eval or require ruby:

source <(sed -E -n 's/[^#]+/export &/ p' ~/.env)

This adds export, keeping comments on lines starting with a comment.

.env contents

A=1
#B=2

sample run

$ sed -E -n 's/[^#]+/export &/ p' ~/.env
export A=1
#export B=2

I found this especially useful when constructing such a file for loading in a systemd unit file, with EnvironmentFile.


does not suppport multiple lines in OSX
N
Nagev

I have upvoted user4040650's answer because it's both simple, and it allows comments in the file (i.e. lines starting with #), which is highly desirable for me, as comments explaining the variables can be added. Just rewriting in the context of the original question.

If the script is callled as indicated: minientrega.sh prac1, then minientrega.sh could have:

set -a # export all variables created next
source $1
set +a # stop exporting

# test that it works
echo "Ficheros: $MINIENTREGA_FICHEROS"

The following was extracted from the set documentation:

This builtin is so complicated that it deserves its own section. set allows you to change the values of shell options and set the positional parameters, or to display the names and values of shell variables. set [--abefhkmnptuvxBCEHPT] [-o option-name] [argument …] set [+abefhkmnptuvxBCEHPT] [+o option-name] [argument …] If no options or arguments are supplied, set displays the names and values of all shell variables and functions, sorted according to the current locale, in a format that may be reused as input for setting or resetting the currently-set variables. Read-only variables cannot be reset. In POSIX mode, only shell variables are listed. When options are supplied, they set or unset shell attributes. Options, if specified, have the following meanings: -a Each variable or function that is created or modified is given the export attribute and marked for export to the environment of subsequent commands.

And this as well:

Using ‘+’ rather than ‘-’ causes these options to be turned off. The options can also be used upon invocation of the shell. The current set of options may be found in $-.


J
Jules Colle

Not exactly sure why, or what I missed, but after running trough most of the answers and failing. I realized that with this .env file:

MY_VAR="hello there!"
MY_OTHER_VAR=123

I could simply do this:

source .env
echo $MY_VAR

Outputs: Hello there!

Seems to work just fine in Ubuntu linux.


If you use such and env file with Docker, your MY_VAR will contain quotes as part of the value :) docs.docker.com/compose/env-file
@kolypto The same would happen with any other command of the higher voted answers. It is just the choice of the example. This is just to show that you can also just source it - is the core idea. The rest of the tricks is for example to cover special signs as well.
J
Jaydeep Solanki

Improving on Silas Paul's answer

exporting the variables on a subshell makes them local to the command.

(export $(cat .env | xargs) && rails c)


Then you can use this (set -a; source dev.env; set +a; rails c) to also have the benefits of sourcing (e.g. script can execute).
j
jwfearn

SAVE=$(set +o | grep allexport) && set -o allexport && . .env; eval "$SAVE"

This will save/restore your original options, whatever they may be.

Using set -o allexport has the advantage of properly skipping comments without a regex.

set +o by itself outputs all your current options in a format that bash can later execute. Also handy: set -o by itself, outputs all your current options in human-friendly format.


I would probably exec env -i bash to clear the existing environment before calling eval if you need to unset variables that are only set within .env.
F
Flavien Volken

The shortest way I found:

Your .env file:

VARIABLE_NAME="A_VALUE"

Then just

. ./.env && echo ${VARIABLE_NAME}

Bonus: Because it's a short one-liner, it's very useful in package.json file

  "scripts": {
    "echo:variable": ". ./.env && echo ${VARIABLE_NAME}"
  }

How about if you have a lot of variables?
@Madeo you can add as many lines as you want, the same way as the line VARIABLE_NAME="A_VALUE"
Warning: this exports these variables into your active shell...
yes, . is a synonym for source and therefore does the same
g
glen

Here's my variant:

  with_env() {
    (set -a && . ./.env && "$@")
  }

compared with the previous solutions:

it does not leak variables outside scope (values from .env are not exposed to caller)

does not clobber set options

returns exit code of the executed command

uses posix compatible set -a

uses . instead of source to avoid bashism

command is not invoked if .env loading fails

with_env rails console

You can also run inline (the variables are exposed to your current terminal session): set -a && . ./.env && "$@" && echo "your comand here"
g
gogo

If env supports the -S option one may use newlines or escape characters like \n or \t (see env):

env -S "$(cat .env)" command

.env file example:

KEY="value with space\nnewline\ttab\tand
multiple
lines"

Test:

env -S "$(cat .env)" sh -c 'echo "$KEY"'

J
Javier Buzzi

Simpler:

grab the content of the file remove any blank lines (just incase you separated some stuff) remove any comments (just incase you added some...) add export to all the lines eval the whole thing

eval $(cat .env | sed -e /^$/d -e /^#/d -e 's/^/export /')

Another option (you don't have to run eval (thanks to @Jaydeep)):

export $(cat .env | sed -e /^$/d -e /^#/d | xargs)

Lastly, if you want to make your life REALLY easy, add this to your ~/.bash_profile:

function source_envfile() { export $(cat $1 | sed -e /^$/d -e /^#/d | xargs); }

(MAKE SURE YOU RELOAD YOUR BASH SETTINGS!!! source ~/.bash_profile or.. just make a new tab/window and problem solved) you call it like this: source_envfile .env


I had to read .env text from gitlab secret variable for a pipeline: Based on your solution this command worked for me: source <( echo $(sed -E -n 's/[^#]+/ &/ p' <(echo "${2}" | tr -d '\r')) );. Somehow gitlab saves the secret variable with a windows carriage return, so I had to trim that with tr -d '\r'.
C
Community

I work with docker-compose and .env files on Mac, and wanted to import the .env into my bash shell (for testing), and the "best" answer here was tripping up on the following variable:

.env

NODE_ARGS=--expose-gc --max_old_space_size=2048

Solution

So I ended up using eval, and wrapping my env var defs in single quotes.

eval $(grep -v -e '^#' .env | xargs -I {} echo export \'{}\')

Bash Version

$ /bin/bash --version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin18)
Copyright (C) 2007 Free Software Foundation, Inc.

K
Koichi Nakashima

Use shdotenv

dotenv support for shell and POSIX-compliant .env syntax specification
https://github.com/ko1nksm/shdotenv

eval "$(shdotenv)"

Usage

Usage: shdotenv [OPTION]... [--] [COMMAND [ARG]...]

  -d, --dialect DIALECT  Specify the .env dialect [default: posix]
                           (posix, ruby, node, python, php, go, rust, docker)
  -s, --shell SHELL      Output in the specified shell format [default: posix]
                           (posix, fish)
  -e, --env ENV_PATH     Location of the .env file [default: .env]
                           Multiple -e options are allowed
  -o, --overload         Overload predefined environment variables
  -n, --noexport         Do not export keys without export prefix
  -g, --grep PATTERN     Output only those that match the regexp pattern
  -k, --keyonly          Output only variable names
  -q, --quiet            Suppress all output
  -v, --version          Show the version and exit
  -h, --help             Show this message and exit

Requirements

shdotenv is a single file shell script with embedded awk script.

POSIX shell (dash, bash, ksh, zsh, etc)

awk (gawk, nawk, mawk, busybox awk)


Awesome tool and great attention to details. Thanks!!
@PierreGramme Using a dedicated tool running at least two forks for resolving a problem of understanding a concept reduced in one command seem a little overkill!
@FHauri Maybe an overkill, but this question has 43 different answers: was it really such a simple problem? In my use case I have a .env file written in Python dialect and apply it to Bash. Can't simply use source due to different conventions for managing spaces etc. That tool and its analysis of differences was definitely useful for me
b
b4hand

You can use your original script to set the variables, but you need to call it the following way (with stand-alone dot):

. ./minientrega.sh

Also there might be an issue with cat | while read approach. I would recommend to use the approach while read line; do .... done < $FILE.

Here is a working example:

> cat test.conf
VARIABLE_TMP1=some_value

> cat run_test.sh
#/bin/bash
while read line; do export "$line";
done < test.conf
echo "done"

> . ./run_test.sh
done

> echo $VARIABLE_TMP1
some_value

Unlike most other answers, this solution doesn't eval test.conf as a script file. That makes it better. It's safer to not allow scripting unless you actually need it, especially if someone don't realize that's what's going on (or forgets).
A
Alex Skrypnyk
t=$(mktemp) && export -p > "$t" && set -a && . ./.env && set +a && . "$t" && rm "$t" && unset t

How it works

Create temp file. Write all current environment variables values to the temp file. Enable exporting of all declared variables in the sources script to the environment. Read .env file. All variables will be exported into current environment. Disable exporting of all declared variables in the sources script to the environment. Read the contents of the temp file. Every line would have declare -x VAR="val" that would export each of the variables into environment. Remove temp file. Unset the variable holding temp file name.

Features

Preserves values of the variables already set in the environment

.env can have comments

.env can have empty lines

.env does not require special header or footer like in the other answers (set -a and set +a)

.env does not require to have export for every value

one-liner


I really appreciate your solution. In most projects you have .env files with comments, spaces, no export statement etc. Pretty nice!
V
Victor Roetman

Building on other answers, here is a way to export only a subset of lines in a file, including values with spaces like PREFIX_ONE="a word":

set -a
. <(grep '^[ ]*PREFIX_' conf-file)
set +a

T
Tudor Ilisoi

My .env:

#!/bin/bash
set -a # export all variables

#comments as usual, this is a bash script
USER=foo
PASS=bar

set +a #stop exporting variables

Invoking:

source .env; echo $USER; echo $PASS

Reference https://unix.stackexchange.com/questions/79068/how-to-export-variables-that-are-set-all-at-once


A
Alex

My requirements were:

simple .env file without export prefixes (for compatibility with dotenv)

supporting values in quotes: TEXT="alpha bravo charlie"

supporting comments prefixed with # and empty lines

universal for both mac/BSD and linux/GNU

Full working version compiled from the answers above:

  set -o allexport
  eval $(grep -v '^#' .env | sed 's/^/export /')
  set +o allexport

what's the point of "-o allexport" if you prepend them with "export" anyway?
l
lapsus63

My version :

I print the file, remove commented lines, emptylines, and I split key/value from "=" sign. Then I just apply the export command.

The advantage of this solution is the file can contain special chars in values, like pipes, html tags, etc., and the value doesn't have to be surrounded by quotes, like a real properties file.

# Single line version
cat myenvfile.properties | grep -v '^#' | grep '=' | while read line; do IFS=\= read k v <<< $line; export $k="$v"; done

# Mutliline version:
cat myenvfile.properties | grep -v '^#' | grep '=' | while read line; do 
  IFS=\= read k v <<< $line
  export $k="$v"
done


G
Guss

I have issues with the earlier suggested solutions:

@anubhava's solution makes writing bash friendly configuration files very annoying very fast, and also - you may not want to always export your configuration.

@Silas Paul solution breaks when you have variables that have spaces or other characters that work well in quoted values, but $() makes a mess out of.

Here is my solution, which is still pretty terrible IMO - and doesn't solve the "export only to one child" problem addressed by Silas (though you can probably run it in a sub-shell to limit the scope):

source .conf-file
export $(cut -d= -f1 < .conf-file)

P
Polv

Modified from @Dan Kowalczyk

I put this in ~/.bashrc.

set -a
. ./.env >/dev/null 2>&1
set +a

Cross-compatible very well with Oh-my-Zsh's dotenv plugin. (There is Oh-my-bash, but it doesn't have dotenv plugin.)


I
Iwan Aucamp

I use this:

source <(cat .env \
  | sed -E '/^\s*#.*/d' \
  | tr '\n' '\000' \
  | sed -z -E 's/^([^=]+)=(.*)/\1\x0\2/g' \
  | xargs -0 -n2 bash -c 'printf "export %s=%q;\n" "${@}"' /dev/null)

First Removing comments:

sed -E '/^\s*#.*/d'

Then converting to null delimiters instead of newline:

tr '\n' '\000'

Then replacing equal with null:

sed -z -E 's/^([^=]+)=(.*)/\1\x0\2/g'

Then printing pairs to valid quoted bash exports (using bash printf for %q):

xargs -0 -n2 bash -c 'printf "export %s=%q;\n" "${@}"' /dev/null

Then finally sourcing all of that.

It should work for just about all cases with all special characters.


A
Anand Tripathi

First, create an environment file that will have all the key-value pair of the environments like below and named it whatever you like in my case its env_var.env

MINIENTREGA_FECHALIMITE="2011-03-31"
MINIENTREGA_FICHEROS="informe.txt programa.c"
MINIENTREGA_DESTINO="./destino/entrega-prac1"

Then create a script that will export all the environment variables for the python environment like below and name it like export_env.sh

#!/usr/bin/env bash

ENV_FILE="$1"
CMD=${@:2}

set -o allexport
source $ENV_FILE
set +o allexport

$CMD

This script will take the first argument as the environment file then export all the environment variable in that file and then run the command after that.

USAGE:

./export_env.sh env_var.env python app.py

L
Lucas B

How I save variables :

printenv | sed 's/\([a-zA-Z0-9_]*\)=\(.*\)/export \1="\2"/' > myvariables.sh

How I load them

source myvariables.sh