How to update Magento 2 with real world example

Every Magento developer faces the time to update a magento installation to the latest version. And every Magento developer says it was harder than it looked like at the beginning.

We will see an “utopic world” example and then a “real world” example. Both from Magento 2.3.3 (I explain why later) to the 2.4.6-p3 version that is the current latest version.

Why is it so hard?

Well, Magento is a Framework/CMS that is always pushing its limits to the most recent versions of the technology it uses and it’s always trying to incorporate new and better technology.

So it’s not just as running an “update” command… You need to install new software (Elasticsearch for example), change your database (export and import to do it), run some composer commands in certain order to properly work… So you may need some sysadmin knowledge or someone that helps you with this part.

As I use Devilbox for this kind of things in my local computer, I have no problems. I normally use docker-magento for day to day use, but I am most used to Devilbox when it needs some system modifications.

This is what I thought doing in first place with this post

I wanted to write a post where I would install a Magento 2.0.0 and then start updating it… but it would be too much work haha.

And then in my job, I had to update a Magento 2.3.3. So I decided to write this post instead, telling what I had to do step by step… A real world example.

Sometime in the future, I will try the same as in this post but from Magento 2.0.0 just to try and see if Magento is “updateable” from any version to the latest one or I have to make some steps in between to middle versions (hope not).

What is this post about then?

So I thought it would be nice to describe the journey of updating it step by step and hopefully it would be a helpful to another developer sometime.

It’s more real, as it was a real production web, with real data and third party modules and a third party theme, etc. With, let’s say, real problems.

Context of the update

This was a Magento 2.3.3:

  • PHP 7.3
  • MySQL 5.7
  • Composer 1 (1.something… I don’t remember the exact version right now)
  • No Redis, No Varnish… so at least I did’t have to deal with sessions migration, but honestly I would suggest to delete sessions here and start from scratch.
  • Not too much products (~400)
  • 4 stores, everyone with a different domain and different languages each
  • ~20 third party modules (some in app/code, some others in vendor)
  • using third party theme that had ~10 modules in app/code
  • 2 child themes (here we had some bad practices, as we duplicated code between themes instead of creating a common one to inherit… but what can I say? Sometimes the customer doesn’t let you work as well as you would like)
  • No strange payment methods
  • 1 integration in checkout, implemented with observers
  • No direct modifications in app/code or in vendor (at least we didn’t have used forbidden techniques here)

As you can see it’s a more or less old version, a not supported anymore one, that didn’t have a requirement for Elasticsearch yet, not too much products. And not too much bad practices applied. It should not be a difficult one so it’s a good example to learn.

Preparation/Planification

There are two main links we must read first. They will have (mostly… in real situations you may need to make some research) any instructions to perform it correctly:

You first need to know the version you are moving your Magento to. Because sometimes, you won’t be able to move to the final version and maybe you prefer to reach a supported version that it’s not the latest one, because of some important module that is not yet compatible with (in this case) PHP 8.2. So you have to be happy with the latest your instance admit.

Backup

Take a full backup (files and database) before doing any change. I like to enable maintenance mode and then make the copy, so I can be sure that no orders or any other change are taking place.

Standard and “not real” steps

IMPORTANT NOTE: Let’s see first how you would do these steps if the world was ideal and utopic, and your Magento was the simplest Magento instance you can find. These are the strictly necessary steps to update your instance BUT it does not have third party modules, third party themes, or anything else. It’s just for you to see the basic steps. Then we are going to complete these steps with the “real” ones. This is just for making it from simpler steps to more complicated steps.

Prepare for update

Maintenance mode on, deploy mode set to developer, stop crons, finish queues

bin/magento maintenance:enable && bin/magento deploy:mode:set developer && bin/magento cron:remove && bin/magento cron:run --group=consumers

Composer root plugin

composer-1 require magento/composer-root-update-plugin ~2.0 --no-update

This was introduced so that it can automatically make some changes in composer.json through the new require-commerce command. Nothing else needed is needed to know from here.

Change server versions

DATABASE

In my case, export the database that was a MySQL 5.7 and import it in a new one but with MySQL 8.0.

Same name, same users, same everything if possible.

NOTE: in this case, if it is a production website, you may need to make some repaces in the database for any instances of the “definer” that could be present. But only if you change your database user. For this I use this command:

sed -i 's\DEFINER=`old_db_user/DEFINER=`new_db_user/g' your_db_dump.sql

And if you don’t know your definer just make (you may find several… coming from bad migrations, time to solve that now):

cat your_db_dump.sql | grep 'DEFINER='

THE REST

As we are going to Magento 2.4.6-p3 as this is the latest version the time I’m writing this.

https://experienceleague.adobe.com/docs/commerce-operations/installation-guide/system-requirements.html

  • PHP 8.2
  • MySQL 8.0 (or MariaDB 10.6)
  • Elasticsearch 7.17 (or OpenSearch 2.5)
  • No Redis, No Varnish in my case
  • Composer 2.2, but be careful, we need to execute first commands with Composer 1 yet

TIP: I like to check compose.json for exact restriction here https://github.com/magento/magento2/blob/2.4.6-p3/composer.json. Just because there you find more explicit version restrictions and you may find some surprises like “composer/composer”… it can’t be 2.2.16… it seems that that exact version had some bug.

Take a look at app/etc/env.php for some values you may need to change (db connection, session save_path, etc). I prefer to disable Redis and Varnish if they are enabled to avoid cache conflicts in this point.

Update your Magento instance

NOTE: the composer require command for this step has changed, take a look at the documentation with the instructions we already said above (this one –> https://experienceleague.adobe.com/docs/commerce-operations/upgrade-guide/implementation/perform-upgrade.html)

NOTE2: composer v1 yet here!!

Composer commands

composer require-commerce magento/product-community-edition 2.4.6-p3 --no-update
composer update

Update in composer.json some metadata, it’s not required but let’s keep good practices:

  • “name”: “magento/magento2ce”
  • “version”: “2.4.6-p3”
  • “description”: “Magento 2 (Open Source)”

Magento commands

bin/magento setup:upgrade

Now you should run setup:upgrade and it should work but, only if you have your Elasticsearch reachable in default connection settings… If any of those change, setup:upgrade will fail as it cannot reach Elasticsearch.

You have some options here, being the easiest the third one. But let’s use the second, as you may not need to do it unless setup:upgrade fails because of Elasticsearch:

  • Run some MySQL commands to insert in core_config data the valules required (as the second option will fail if you run them before trying setup:upgrade). I don’t like this one…
  • Run core:config:set some values instead of MySQL insert statements. Just be sure to first fail the setup:upgrade as it will insert wrong values in core_config_data but those values will exist, so core:config:set can update them. Those are MY values, change them to your own ones:
    • bin/magento config:set catalog/search/engine elasticsearch7
    • bin/magento config:set catalog/search/elasticsearch7_server_hostname 172.16.238.240
    • bin/magento config:set catalog/search/elasticsearch7_server_port 9200
    • bin/magento config:set catalog/search/elasticsearch7_index_prefix mage246
    • bin/magento config:set catalog/search/elasticsearch7_enable_auth 0
    • bin/magento config:set catalog/search/elasticsearch7_server_timeout 15
    • bin/magento config:set catalog/search/elasticsearch7_minimum_should_match “”
  • Put the values in app/etc/env.php and run setup:upgrade (the easiest one)

If this has failed because of Elasticsearch, retry setup:upgrade, it should now work.

bin/magento setup:di:compile
bin/magento setup:static-content:deploy -f

Finish steps

Revert the changes in the “prepare for update” section and reindex (here we are reseting indexes but first setting them to update on save, and finally update on schedule again as it ir safer to do in that way).

bin/magento indexer:set-mode realtime && bin/magento indexer:reset && bin/magento indexer:reindex && bin/magento indexer:set-mode schedule

bin/magento cron:install && bin/magento dep:mode:set production

bin/magento maintenance:disable

A “more real” example

Let’s do the same steps, but adding some more in between them… because in real world scenarios you have app/code modules, third platforms integrations you have to test, weird and strange module dependencies (vendor_core module in app/code but all the other vendor_module modules in vendor… yes, you have composer broken dependencies now), duplicated urls that break the process, etc

Duplicated url rewrites

In version before 2.4.3, there were no primary keys in the url rewrite table… so if could happen that you had duplicated url rewrites so when you run your setup:upgrade in the new version, it will yell at you with an error of MySQL “Integrity constraint violation: 1062 Duplicate entry”.

Well, don’t panic. It can be easily solved.

In my case was in the catalog_url_rewrite_product_category table.

You need to use some modules that help with this but be careful as some of them will just “unduplicate” them adding some suffix to the url or just reseting them. Something that could solve your duplicated urls problem and lead to new SEO problems (lots of 404 of the old google indexed urls).

In my case I could just dump those db entries, delete them, and reimport them without duplication.

I put this step here in the beginning, but you can do it when your setup:upgrade fails.

In the worst scenario possible here… you have to put off the update until someone (usually marketing) tell you the new urls. So you have to just export them somehow (with some SQL statement), and then when they are fixed and given back to you, just delete them and import the fixed ones.

Composer root plugin

This step is the same as above, nothing new.

Change server versions

This step is the same as above, nothing new. Just remember, still Composer 1 here.

Update your Magento instance

This step is the same as above, nothing new.

Just a difference. Don’t run setup:upgrade or any setup:di:compile as you haven’t updated modules yet, so it will probably fail.

Just do the “composer commands” section above. We will do the “magento commands” later when we already have updated modules and theme.

Update your modules

app/code to vendor

As a substep I would add to change any app/code module to vendor if it’s possible. It’s not the first time that you work on a web where you see that there are some modules that were not installed using composer, but they do have this installation option.

Why?

Maybe it didn’t work correctly in that moment, repo was down and the alternative was putting the code in app/code directory… but now it’s a good moment to fix this.

The thing is that you can throw any module in this directory and no one will complain. Composer will complain if the required module does not have all the dependencies satisfied.

How?

Just disable the modules, delete them from app/code (back it up just in case), and require the module using composer. Then enable it again.

From the Magento perspective, nothing has changed.

Be careful

Something to be careful here is that they might (but shouldn’t) be modifications directly to that module… and you will delete it and reinstall it taking away all bad direct modifications to that code.

Composer modules

IMPORTANT NOTE HERE: You have to update magento instance with Composer 1, but then after that (in this step) you should use Composer 2 from now on. Or you may find some package not found errors. So this step must be run with Composer 2.

Update composer modules. Maybe you don’t need to do this step. I mean that a simple composer update will bring the latest versions, but maybe the type of installation you or someone else did in the past restricts the download of the latest version.

Just take a look at the composer.json file to see if there are some weird version restrictions. You should not see something like this in the “require” node: “some dependency here”: “1.4” but something like this “some dependency here”: “^1.4”

If that’s the case, run composer require command again on that module.

And maybe it’s a good idea to pick some random modules of some vendor and see if after the composer update command in later steps you actually have the latest version.

If someone messed up with hard restictions, you may find a hard time trying to solve the puzzle: your module needs another module that is also hard versioned. So go with that other module first.

app/code modules

Update any other module here following the vendor instructions (normally just dropping the files and running magento commands we will run at the end of all the process).

The modules that came with the theme, you will probably have them in the updated theme files (next step)

Update your theme

Two options here. You have a Luma child theme or you have a third party theme.

Luma child theme

You have some tools to automate this task, but maybe it’s not so time consuming to do it by hand…. you need to compare the child theme files with the updated Luma theme.

You need to find those overriden files in your child theme, that have important changes to be incorporated from the updated file in the Luma theme.

For example: a new variable was instroduced in some phtml, instead of having inline code… so you need to incorporate that change to your child theme file.

For this task I use Meld (a linux base OS tool to compare files and directories as a git diff), and take a look for the changes and see if the change needs to be incorporated or not.

Third party module

You have to follow theme update instructions (normally just dropping new files in app/design/frontend and app/code, and maybe some in pub/ and run magento commands we will run at the end of the process).

And now repeat the same comparison between your child theme files and the updated third party module files, to see if there are changes because of the update that need to be incorporated to your overriden files.

Update other changes made

Here we are usually talking about the modules the developers of the web develop for customizations the customer asks for.

Those are not “updateables” ones, as there is nowhere to pick an updated version… it’s just code.

Check for Overrides

There are some tools to automate this step too, but I did it manually as I didn’t have a lot of modifications made.

You need to find any overrides in your modules, we are usually talking about the web developer’s modules that were not updated in before steps.

When found, you need to compare your file (the one that is overriding) with the updated overriden file to see if there are any changes that need to be incorporated to your file.

Check for PHP changes

Again, there are some tools that let you know if your PHP code needs a change to fit the new PHP version.

And again, I did this by hand just testing the module works properly at the end of the process or if it throws any error to the page. If that’s the case, solve the error as it usually is deprecation of some function calls with empty values, etc. They are not hard to solve googling the error.

“magento commands” of the Update your Magento instance section

Time to run those commands. Nothing should fail, but don’t worry: it will probably fail hahaha

Time to solve those little mistakes we made (a module that did not update properly, a url that was still duplicated, etc…).

When the commands run properly we have updated everything.

Finish steps

Nothing changes here, same as above section.

Test

You should probably test everything… or at least the most important parts (make a purchase, pay it, create a user, log in, send contact form, etc).

But there are also some functionallities that may be hard to test as they are integrations with third party services… Here I had a checkout integration, where I send the order to another system.

So you may need to change the API keys of the that integration, and test it to see if data still travels as you wish to the external service and then put production keys again.

Bugs

What do yo do if you find a bug in a core magento file? Well… fix it.

You decide how, but I usually make a vendor modification as it is a “hotfix” for the upgrade to work. And then with more time, let’s try to find out a better solution rather than changing a vendor file.

We don’t have time right now to see what the problem is, and how it was caused so we can find an observer that fixes it in a new module, etc… We need our update done first and we will later work on the bug.

So make the modification, copy the file to a new directory with the same directory structure and then I like to make a simple bash script that just copies those files to the mageto installation so they get changed.

If someone downloads the project from the repo, I put a teporary note that after getting the project running, they have to run this script.

Then if we found a better way to solve it, just remove the script and the temporary note 🙂

DNS changes

You probably want to make all these changes in a new server, where you moved all the files and the database and you are accessing it by setting that server IP to your hosts file (as you should do). I mean, you leave the production server in maintenance mode, and don’t touch anything there.

Then copy all the files to the new server, make here all the process and finally just change the DNS of the domain to point the new server.

This way you have your production “not updated” web untouched and backed up, and your “updated” web in another server. Everyone’s happy.