Jump to content


Photo
- - - - -

Hardcoded domain and server urls should not be stored in serialized db fields

DMS2

  • Please log in to reply
6 replies to this topic

#1 cemico

cemico

    Advanced Member

  • Members

  • 62 posts
  • Country: Country Flag

Posted 13 May 2014 - 08:08 PM

Today i wanted to migrate my pagelines site to a new testdomain, where i can test the updating of the DMS2, Wordpress and all plugins.

 

Unfortunately my process of migrating Wordpress sites, like i'm doing it for many years now, did not work.

 

My process is:

- make an MySQL dump from the main site

- copying the wordpress folder from the main site

- create a new subomain for the test site

- create a new wordress test folder on the test server

- upload the copied FTP files to the new wordress test folder

- import the MySQL dump in a new test database on the server

- change all urls in the database to point to the new test domain (via a normal "Find and Replace" in a text editor)

 

This will lead to losing a ton of settings, templates and content due to this problem:

 

DMS stores serialized data in the database.

In this serialized data you can find hardcoded base urls and hardcoded filesystem urls.

 

An example is:

 
"description";s:56:"A responsive and sticky navigation bar for your website.";s:4:"name";s:6:"NavBar";s:8:"base_url";s:68:"http://test.domain.de/wp-content/themes/dms/sections/navbar";s:8:"base_dir";s:59:"/usr/www/users/test/wp-content/themes/dms/sections/navbar";s:9:"base_file";s:71:"/usr/www/users/test/wp-content/themes/dms/sections/navbar/section.php";s:9:"workswith"

Here you can find two issues:

 

- Never(!) store the base url like http://test.domain.de in the database.

=> The only fields which should contain the base url should be wordpress' own fields in the option table. The site url and the home url

 

- Never(!) store filesystem urls like /usr/www/users/test in the database

 

If you are storing urls in the db and you are reading from them, especially as DMS is doing it with serialized data, you get real pain when migrating a site via the normal MySQL dump/FTP dump migration process.

 

The problems are:

1. base urls like http://test.domain.de in serialized data

- Whenever the base url changes, maybe due to a customer renaming his brand, and the home and site url is changed in Wordpress, the whole DMS breaks if you don't update your database via an additional replacement script.

- When migrating: You really do have to replace all urls in your sql dump via a script, that knows how to replace serialized data.

For all you out there facing this issue - I found this tool, which works like a charm:

https://interconnect...ress-databases/

 

2. filesystem urls like /usr/www/users/test

- When migrating: You really do have to remember to change all stored filesystem urls to the new filesystem urls on the new server.
Firstly you have to find out the new filesystem url.
If you forget it, DMS breaks

 

I don't know, why DMS stores the base url additionally. The base url, or "home" is already perfectly present in the Wordpress options table and should be used centralised by any plugin or theme out there. E.g. by using home_url()

If one is storing the base url or even the filesystem urls additionally, like DMS is doing it, it's getting a pain when trying to migrate. And maybe not only when you are migrating...

 

This is not only an idea - It's a feature request which is essential in my opinion.



#2 Simon_P

Simon_P

    Messer

  • Administrators



  • 8388607 posts
  • LocationDevon
  • Framework Version:2.1.1
  • Country: Country Flag

Posted 14 May 2014 - 11:18 AM

Looks like the snippet you quoted there is from the cached section data, by clicking the flush caches button in the DMS settings all those urls would have been rebuilt.

 

IF you insist on editing sql files directly then either develop the site locally using the same URL with a tool such as MAMP like in my howto on the blog OR use a URL with the exact same number of characters, then you can copy/paste as usual.



#3 cemico

cemico

    Advanced Member

  • Members

  • 62 posts
  • Country: Country Flag

Posted 14 May 2014 - 11:22 AM

@Simon_P

I understand your "how to" and "why".

 

But i really do think, that you never should have the base or filesystem urls in the db. Not in a cache table and nowhere else.

This only leads to problems.

Those urls can and should always be read centralized via wordpress variables and/or functions or php functions.



#4 Simon_P

Simon_P

    Messer

  • Administrators



  • 8388607 posts
  • LocationDevon
  • Framework Version:2.1.1
  • Country: Country Flag

Posted 14 May 2014 - 11:28 AM

They are there because they need to be, to include php files you need absolute urls and paths to files, thats how the sections work. I'm sorry your years old method of simply editing the sql file has to be modified but thats the way it is.

If you use some of the methods suggested you will never have to edit any sql files.



#5 cemico

cemico

    Advanced Member

  • Members

  • 62 posts
  • Country: Country Flag

Posted 14 May 2014 - 11:44 AM

@Simon_P

Ok, i'll try to make a code review and check how and why you have to store absolute paths to include files in a database.

 

In the years of developing php applications, for me, there was never a need to do this, and i didn't see it like this anywhere.

I mean: Why do one have to store this filesystem path, pointing to the server root directory, in a database?
/usr/www/users/test
This can be read easily via PHP variables and you could provide a placeholder variable in the database which gets replaced after reading...

 

"My years old method" of migrating sites" is really the method it "normally" has to be. And i know many developers who'd sign that and who are doing it exactly that way.

 

But the best thing is, that i know now, because of your tipps, how to migrate wordpress pages in the one specific case when they are setup with DMS.

And with the tool i mentioned, this is really easy. One has only to remember it :)



#6 faustus501

faustus501

    Newbie

  • Members

  • 8 posts
  • LocationSydney
  • Country: Country Flag

Posted 29 May 2014 - 02:07 AM

How do you migrate a site and have urls that are in sections eg text box automatically change to the new domains url?

 

Have tried a manual find and replace in sql and also have tried using Migrate DB which did everything well but not the urls in sections that I have used to build templates. Are these urls not in the database? if not where are they?

 

Also where are the template files stored? If I can get into these perhaps I can to a find and replace here? Or do you have to export the DMS config and change them in the .json file and then import that into migrated site?

 

I am having real difficulty with this so any help would be great.

 

Thanks



#7 Danny

Danny

    Is Awesome!

  • Moderators
  • 17015 posts
  • LocationManchester, UK
  • Country: Country Flag

Posted 29 May 2014 - 11:31 AM

Not sure I understand what you mean ?

 

Are you saying you have added a URL in TextBox using HTML ? If so these URLS will not dynamically change based on the domain they're used in as you have inputted them yourself.

 

If I have misunderstood, please create your own topic and we can assist you further. If you can provide screenshots it will help us understand your problem.







Also tagged with one or more of these keywords: DMS2