Deploy Magento 2 with short downtime - NicolasBlancoM

I have done many tasks for a web in Magento 2 that had 3 websites, 3 languages each website and the active theme was a child of a child… The deployment process took 15 to 20 minutes. This is unacceptable for a production site.

How can we shorten this time? Let’s see it.

TL;DR ok, just go to this section above “Example: Deployment script explained” you have the code explained step by step

The problem with “normal” deployment process

When making changes in a Magento 2 site, we normally run three commands that make your site to stop working the entire execution time:

bin/magento setup:upgrade && bin/magento setup:di:compile && bin/magento setup:static-content:deploy

Or the short way

bin/magento s:up && bin/magento s:d:c && bin/magento s:s:d

The first one s:up takes normally less than a minute. The other two take several minutes. Being the third s:s:d the one that can take 15 minutes (it deploys each theme, for each language).

Solution to shorten downtime

Since Magento v2.2, a “pipeline deployment” process was introduced to solve this.

You can read the documentation here https://experienceleague.adobe.com/en/docs/commerce-operations/configuration-guide/deployment/overview, but let’s simplify it by taking out of the explanation everything that does not have direct relation with this (I didn’t understand, the first time I read it, why I needed two environments, why I needed two different .gitignore files, why I needed to copy all those files, why I needed to run app:config:dump… lots of questions I know).

Simplified process

If you read the docs above, you see that they recommend three environments, the use of git in production, push changes to git and then pulling… many different concepts are mixed. Let’s just talk about what exactly speeds up the deployment process.

Basic process

The basic idea, is to run the second and third commands seen above (s:d:c and s:s:d) in “another place than production site”. That allows:

our production site to keep working meanwhile the commands are executing
copy resultant files to production site from this “other place” where we executed commands (copy those files is much much faster than having to generate them)
finally, execute the first command (s:up) in production –> this will be the only downtime and it takes 1 minute more or less

What is this “other place”?

It can be just another directory of the same server. You can run all the process in that directory without stopping production site.

In order to understand what is going on and how it works, so we can then modify the process as we need it, let’s see a script that automates the process of a production site deployment that is stopped just for 20 seconds.

Example: Deployment script explained

This is a executable bash script placed outside the public directory of our server

/path/to/production/site/htdocs -> magento app is here
/path/to/production/site/bin -> script is placed here
/path/to/production/site/deployment -> deployment process takes place here, this is the “other place”

Let’s explain step by step

#!/bin/bash

#1 variables
curPath=$(pwd)
magePathParentDir="htdocs" # dir name where magento app is placed
magePath="$curPath/../$magePathParentDir" # full path to $magePathParentDir
deployDir="deployment" # dir name where deoployment process will take place
deployPath="$curPath/../$deployDir" # full path to $deployDir

#2 execute bin/magento app:config:dump on production
cd $magePath
php bin/magento app:config:dump

#3 move to mage dir, create deployment dir by copying htdocs
echo "... copying production files to deployment dir ..."
cd ..
mkdir $deployDir 2> /dev/null
cp -a "$magePathParentDir/." $deployDir

#4 move to deployment dir and remove unnecessary files
cd $deployPath
rm -f ./app/etc/env.php

#5 execute compile and deploy on deployment dir
echo "... compile and deploy on deployment dir ..."
cd $deployPath
php bin/magento s:d:c && php bin/magento s:s:d -f

#6 enable magento maintenance and remove production pub/static/ and generated/ dirs
echo "... starting shadow deployment on production ..."
cd $magePath
php bin/magento main:ena
rm generated/* pub/static/* -rf

#7 copy deployment pub/static/ and generated/ dirs to production
echo "... copying deployment files to production ..."
cp -a "$deployPath/generated/." "$magePath/generated"
cp -a "$deployPath/pub/static/." "$magePath/pub/static"

#8 execute setup upgrade, clean cache on production
echo "... executing deployment commands on production ..."
cd $magePath
php bin/magento app:config:import && php bin/magento setup:upgrade --keep-generated && php bin/magento cache:flush

#9 disable magento maintenance
cd $magePath
php bin/magento maintenance:disable

#10 clean deployment files
echo "... cleaning deployment files ..."
rm -r "$deployPath"

echo "... all done ..."

Step #1

It defines some variables to let the script know where it is.

htdocs is my server public dir as I am in a local environment using apache for this example

bin is the dir that holds this code, and it’s at the same level as htdocs

deployment is where the deployment process will take place, and it is at the same level as htdocs as well

Step #2

This is the step that allows magento to execute the process without database connection (for example, it lets the CLI know what themes are installed without quering the database). It dumps database info into app/etc/config.php

Step #3

It copies all the production site from htdocs dir, to deployment dir. It is not necessary to copy all files, this is just an example that works. Not all pub/media would be needed, but let’s keep it simple and copy all files, they will be removed soon.

Step #4

Let’s just make sure this magento copy cannot access production database by removing this deployment copy app/etc/env.php file… Just in case

Step #5

Being at the deployment dir, execute compile and deploy commands (the second and third commands explained above, s:d:c and s:s:d). This will get the needed database info by reading it from app/etc/config.php file, that’s why this works. Otherwise, it will not know what themes it has to deploy or what languages you have…

Now you have a fresh generated/ and a fresh pub/static/ dirs that have been created without having to stop your web.

Now you just need to copy them to production site, and run the only command that needs to be executed in production: s:up

Step #6: This is the first step that has the site stopped

So we now stop production site by enabling maintenance mode, and just to be sure, remove production generated/ and pub/static/ dirs as they will be copied from the deployment dir

Step #7

Copy generated/ and pub/static/ dirs from the deployment dir

It is also very quick, just a few seconds. We have generated them in “another place” (the deployment dir) so we can just copy them and spend less time with the site down for maintenance

Step #8

Execute database update in magento, the only command required to be executed on the production site (not a copy).

It usually takes less than a minute, but let’s say it takes 2 minutes max

It’s important the app:config:import command as in a real scenario we are moving across servers.

In this example we copied production site to another dir of the same server so it wouldn’t be needed as we are basically just trying to import into the database, what we exported to app/etc/config.php a few steps above.

But in a real scenario, we would have done app:config:dump (step #2) in another server with lots of changes made in that environment, and we are dumping that databse info in the app/etc/config.php file that will need to be copied to production site on step #6 (just after enabling maintenance mode) using git or another method. We are not explaining that part here as we are making the most simple scenario.

It’s also important the s:up –keep-generated because s:up without –keep-generated would remove generated/ and pub/static/ dirs we have just copied. And we don’t want that

Step #9 and #10

Just disable maintenance mode, and remove deployment dir.

Your site has been down for a couple of minutes or less. In a fresh Magento installation with some 3rd party modules and 3rd party theme (a ver usual scenario) the downtime should be less than 30 seconds.

More realistic scenario

The above example is just a silly example, where we have a production site and we just need to deploy whatever code we have in that production site but without stopping it for a long time.

Some changes are needed to above steps if we want a more real scenario. You can implement those changes in many different ways, the important thing is to understand what we are really doing and why… the way you do it is up to you.

In step #6 we would actually pull new code to our production site, do it as you wish but that is the spot. We need the latest code having in mind we have already copied generated/ and pub/static/ directories, so don’t make sure you don’t overwrite them in the process of bringing the latest code. Again, do it as you wish, it is not the main part of the explanation.

Another change: we would have also pulled that code in step #3 where we copied the code to deployment dir. So in a real scenario we would copy production to deployment dir and also pull the new code to this deployment dir so we have all the latest code and then generate the code with s:d:c and s:s:d. So we have generated “newer” code with changes, and we have not touched our production site or even stopped it yet.

What about composer changes? I just copy vendor dir and then make sure I update composer autoload to avoid a composer install execution.

Suggested way of handling new code

I don’t have a good way of doing this. I don’t like having git in production environment, which adobe recommends in its documentation. Having a Capistrano or Jenkins pipeline is not always possible. I like more “dirty” solutions that can be done in more scenarios.

So for now, I like copying files and directories with rsync between servers. I just make sure I have everything updated in development environment and then copy all files to production deployment dir (including vendor yeah… just be sure to update composer autoload).

It can take you a couple of tries to configure this kind of scripts, or even having a Jenkins pipeline but then you can be sure it won’t fail and it won’t let your production site down more than a couple of minutes each time you need to deploy.