Export Firestore to dev environment

Sooner or later you need production data in your development environment to test or develop a more rich UI experience based on real life data. There are a few ways of doing this, but this is the easiest especially if your on a windows machine.

You probably already have a persistent data in your dev environment, if not just run the emulators with the following command.

firebase emulators:start --import=./dev_data --export-on-exit

This will import any data you have in the Firestore and Authentication emulators and then import it on the next run. Don’t forget to add ./dev_data to your .gitignore.

Download the data

The data download need to go via a storage bucket. You will probably already have a few buckets if you published your project at any point. I also assume that you have your firebase and gcloud utils installed and authenticated. If not install them and run:

firebase login
gcloud auth login

Then export the data to the default storage bucket. If your on windows do not use the Cloud Firestore Import/Export page in the console. It will add special characters in the filename preventing you from downloading the data. Use the gcloud tool instead.

gcloud firestore export gs://your-project-name.appspot.com/your-choosen-folder-name

Depending on the size of your Firestore this can take some time. When it’s done we can download the data with gsutil.

gsutil -m cp -r gs://your-project-name.appspot.com/your-choosen-folder-name .

The last . means that the data will be downloaded to the current working folder. I put the data in a folder named prod_data inside my project folder. This folder is also added to the .gitignore file to prevent the data from leaving my dev environment. Then I can run my emulators with the downloaded data.

firebase emulators:start --import=./prod_data --export-on-exit

Conclusion

You can easily automate this procedure in a simple powershell script to grab data whenever. Always be careful when working with production data in your dev environment. Make sure that you don’t leak any data via git for example. Also make sure you keep the data encrypted on your workstation and delete it as soon as you don’t need it anymore.

More reading: https://firebase.google.com/docs/firestore/manage-data/export-import

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: