Migration from ((OTRS)) Community Edition to OTOBO

Welcome and thank you for choosing OTOBO! OTRS, ((OTRS)) Community Edition and OTOBO are very comprehensive and flexible in their application. Therefore, every migration to OTOBO requires thorough preparation and possibly some rework.

Please take your time for the migration and follow these instructions step by step.

INFO

After the migration, the data previously available in ((OTRS)) Community Edition 6 is available in OTOBO 10. We do not change any data of the OTRS 6 installation during the migration.

OTOBO Migration Overview

With the OTOBO migration interface, it is possible to use the following migration strategies:

The general migration strategy.
This is the regular way to perform a migration. Many different combinations are supported:
- Server change: Migrate and simultaneously switch to a new application server.
- Separate application and web servers: You have the choice of whether to run the application and database servers on the same host or on a dedicated host each. This choice is independent of the previous setup in OTRS / ((OTRS)) Community Edition.
- Different databases: Migrate from any supported database to another supported database.
- Different operating systems: Switch from any supported operating system to another supported operating system.
- Docker: Migrate to a Docker-based installation of OTOBO 10.
A variant of the general strategy where the database migration is optimized.

Use the ETL-like migration if the source database must not be burdened by increased load or if access to the source database is a bottleneck. In the general strategy, the data is read row by row first from the otrs database and then inserted into the OTOBO database. In this variant, the complete otrs database tables are first exported, then transformed, and then imported into the otobo database.

Migration from an Oracle-based ((OTRS)) Community Edition 6 installation to an Oracle-based OTOBO installation.

This is a special case that is not supported by the general migration strategy. This means that a variant of the optimized strategy must be used.

WARNING

All strategies work for both Docker-based and native installations. But for Docker-based installations, some specifics must be considered. These specifics are covered in the optional steps.

INFO

It is also feasible to clone the ((OTRS)) Community Edition database to the OTOBO database server before the actual migration. This can speed up the general migration strategy.

Migration Requirements

The basic requirement for a migration is that you already have an ((OTRS)) Community Edition or OTRS 6.0.* running and want to transfer both configuration and data to OTOBO.
WARNING
Please consider carefully whether you really need the data and configuration. Experience shows that in many cases, a fresh start is the better option. This is because the previously used installation and configuration was often rather suboptimal. It might also make sense to transfer only the ticket data and switch the basic configuration to OTOBO best practices.
You need a running OTOBO installation to start the migration from!
This OTOBO installation must contain all OPM packages that are installed in your ((OTRS)) Community Edition and that you also want to use in OTOBO.
If you plan to migrate to another server, the OTOBO web server must be able to access the location where your ((OTRS)) Community Edition or OTRS 6.0._ is installed. In most cases, this is the directory _/opt/otrs* on the server where ((OTRS)) Community Edition is running. Read access can be provided via SSH or via file system mounts.
The ((otrs)) Community Edition database must be accessible from the server where OTOBO is running. Read-only access must be granted for external hosts. If access is not possible or if the migration speed should be optimized, then a dump of the database is sufficient.

INFO

If SSH and database access between the servers is not possible, please migrate ((OTRS)) Community Edition to OTOBO on the same server and only then move the new installation.

Step by Step Guide

Step 1: OTOBO Installation

Please start by installing a new OTOBO system. Your old OTRS / ((OTRS)) Community Edition installation will be migrated to this new system.

WARNING

Under Apache, there are pitfalls when running two independent mod_perl applications on the same web server. Therefore, it is recommended to run ((OTRS)) Community Edition and OTOBO on separate web servers. Alternatively, the OTRS configuration should be removed from Apache before OTOBO is installed. Use the command a2query -s and check the directories /etc/apache2/sites-available and /etc/apache2/sites-enabled to see which configurations are currently available and enabled.

After completing the installation, please log in as root@localhost. Navigate to the OTOBO Admin area Admin -> Packages and install all required OTOBO OPM packages.

The following OPM packages and ((OTRS)) Community Edition "Feature Addons" must NOT and should NOT be installed, as these functions are already included in the OTOBO standard:

OTRSHideShowDynamicField
RotherOSSHideShowDynamicField
TicketForms
RotherOSS-LongEscalationPerformanceBoost
Znuny4OTRS-AdvancedDynamicFields
Znuny4OTRS-AutoSelect
Znuny4OTRS-EscalationSuspend
OTRSEscalationSuspend
OTRSDynamicFieldDatabase
OTRSDynamicFieldWebService
OTRSBruteForceAttackProtection
Znuny4OTRS-ExternalURLJump
Znuny4OTRS-QuickClose
Znuny4OTRS-AutoCheckbox
OTRSSystemConfigurationHistory
Znuny4OTRS-PasswordPolicy

The following OTOBO packages have been integrated into OTOBO 10+. This means that they should not be installed in the target system if the target system is OTOBO 10+. - ImportExport

Step 2: Disable SecureMode

After you have installed OTOBO, please log in again to the OTOBO Admin area Admin -> System Configuration and disable the configuration option SecureMode.

INFO

Don't forget to actually deploy the changed setting.

Step 3: Stop the OTOBO Daemon

This is necessary if the OTOBO daemon is actually running. Stopping the daemon is different between Docker-based and non-Docker-based installations.

In the non-Docker case, execute the following commands as user otobo:

bash


# if you are logged in as root

root> su - otobo

otobo> /opt/otobo/bin/Cron.sh stop

otobo> /opt/otobo/bin/otobo.Daemon.pl stop --force

If OTOBO is running in Docker, you just need to stop the daemon service:

bash


docker_admin> cd /opt/otobo-docker

docker_admin> docker-compose stop daemon

docker_admin> docker-compose ps     # otobo_daemon_1 should have exited with code 0

INFO

It is recommended to perform a backup of the entire OTOBO system at this point. If something goes wrong during the migration, you will not have to repeat the entire installation process, but can instead import the backup for a new migration.

TIP

We recommend that you read the chapter OTOBO Backup backup-restore.

Optional Mount /opt/otrs

Often, OTOBO is intended to run on a new server where /opt/otrs is not initially available. In these cases, the directory /opt/otrs on the ((OTRS)) Community Edition server can be mounted into the file system of the OTOBO server. If a regular network mount is not possible, sshfs could be an option.

Optional: sshpass and rsync

This step is only necessary if you want to migrate ((OTRS)) Community Edition from another server and /opt/otrs from the remote server has not been mounted on the OTOBO server.

The tools sshpass and rsync are needed so that migration.pl can copy files via ssh. To install sshpass, please log in as user root on the server and execute one of the following commands:

Install sshpass on Debian / Ubuntu Linux

bash

sudo apt install sshpass

Install sshpass on RHEL/CentOS Linux

bash

sudo yum install sshpass

Install sshpass on Fedora Linux

bash

sudo dnf install sshpass

Install sshpass on OpenSUSE Linux

bash

sudo zypper install sshpass

The same must be done for rsync if it is not yet available.

Step 4: Preparation

INFO

Make sure you also have a valid backup of your OTRS / ((OTRS)) Community Edition system. Yes, we do not touch any ((OTRS)) Community Edition data during the migration, but sometimes even a wrong entry is enough to cause problems.

Now we are ready for the migration. First, we need to make sure that no more tickets are being processed and no users are logging into ((OTRS)) Community Edition:

Please log into the ((OTRS)) Community Edition Admin area Admin -> System Maintenance and add a new system maintenance slot for a few hours. Afterwards, delete all agent and user sessions (Admin -> Sessions) and log out.

Stop all relevant services and the OTRS daemon

Please make sure that no services or cron jobs are running.

bash

su - otrs
/opt/otrs/bin/Cron.sh stop
/opt/otrs/bin/otrs.Daemon.pl stop --force

Delete the caches and operational data

The cached data and the operational data do not need to be migrated. The email queue should already be empty at this point.

bash

su - otrs
/opt/otrs/bin/otrs.Console.pl Maint::Cache::Delete
/opt/otrs/bin/otrs.Console.pl Maint::Session::DeleteAll
/opt/otrs/bin/otrs.Console.pl Maint::Loader::CacheCleanup
/opt/otrs/bin/otrs.Console.pl Maint::WebUploadCache::Cleanup
/opt/otrs/bin/otrs.Console.pl Maint::Email::MailQueue --delete-all

Optional Step for Docker

There are some specifics to consider if your OTOBO installation is running under Docker. The most relevant one: processes running in a Docker container generally cannot access directories outside the container. However, there is an exception: directories that are mounted as volumes into the container can be accessed. Also note that the MariaDB database running in otobo_db_1 (otobo-db-1 in newer docker compose versions) is not directly accessible from outside the container network.

INFO

In the example commands, we assume that the user docker_admin is used for interacting with Docker. The Docker administrator can be either the root user of the Docker host or a dedicated user with the required permissions.

Copy /opt/otrs to the volume otobo_opt_otobo

::: note Container Names In old docker compose versions, the container names are otobo_web_1, otobo_db_1, and otobo_daemon_1. In newer versions, the container names are otobo-web-1, otobo-db-1, and otobo-daemon-1.

:::

In this section, we assume that the OTRS home directory /opt/otrs is available on the Docker host.

There are at least two viable options:

a. Copy /opt/otrs into the existing volume otobo_opt_otobo

b. Mount /opt/otrs as an additional volume

Let's focus on option a. here.

First, we need to find out where the volume otobo_opt_otobo is available on the Docker host.

bash

 otobo_opt_otobo_mp=$(docker volume inspect --format '{{ .Mountpoint }}' otobo_opt_otobo)
echo $otobo_opt_otobo_mp

For safe copying, we use rsync. Depending on your Docker configuration, the rsync command may need to be run with sudo.

bash

rsync --recursive --safe-links --owner --group --chown 1000:1000 --perms --chmod "a-wx,Fu+r,Du+rx" /opt/otrs/ $otobo_opt_otobo_mp/var/tmp/copied_otrs
ls -la $otobo_opt_otobo_mp/var/tmp/copied_otrs  # for checking only
sudo rsync --recursive --safe-links --owner --group --chown 1000:1000 --perms --chmod "a-wx,Fu+r,Du+rx" /opt/otrs/ $otobo_opt_otobo_mp/var/tmp/copied_otrs
sudo ls -la $otobo_opt_otobo_mp/var/tmp/copied_otrs  # for checking only

This copied directory will be available inside the container as /opt/otobo/var/tmp/copied_otrs.

Step 5: Migrate data

:::note Docker Compose Command In old Docker Compose versions, the command docker-compose was used. In newer versions, docker compose is used. :::

Please use the web migration tool at http://localhost/otobo/migration.pl. Note that you may need to replace "localhost" with your OTOBO hostname and possibly add your non-standard port. The application will then guide you through the migration process.

WARNING

Sometimes a warning is displayed that the deactivation of SecureMode was not detected. Please restart the web server in this case. This forces the web server to read the current configuration.

bash

service apache2 restart
cd /opt/otobo-docker
docker-compose restart web
docker-compose ps

INFO

If OTOBO is running in a Docker container, keep the default settings localhost for the OTRS server and_/opt/otobo/var/tmp/copied_otrs_ for the OTRS home directory. This is the path of the data copied in the optional step.

INFO

The default values for the OTRS database user and password are taken from Kernel/Config.pm in the OTRS home directory. Change the suggested settings if you are using a dedicated database user for the migration. Also change the settings if you are working with a database that has been copied into the otobo_db_1 Docker container.

INFO

In the Docker case, a database running on the Docker host is not reachable via 127.0.0.1 from within the Docker container. This means that the setting 127.0.0.1 will not be valid for the OTRS-Server input field. In this case, enter one of the alternative IP addresses reported by the command hostname --all-ip-addresses for OTRS-Server.

INFO

When migrating to a new application server or to a Docker-based installation, the database often cannot be accessed from the target installation. This is usually because the otobo database user can only connect from the host where the database is running. To enable access nonetheless, it is recommended to create a dedicated database user for the migration, e.g., CREATE [USER](../agents/agents.md) 'otrs_migration'@'%' IDENTIFIED BY 'otrs_migration'; and GRANT SELECT, SHOW VIEW ON otrs.* TO 'otrs_migration'@'%';. This user can be deleted again after the migration: DROP [USER](../agents/agents.md) 'otrs_migration'@'%'.

Custom settings in Kernel/Config.pm are transferred from the old OTRS installation to the new OTOBO installation. If you have custom settings, you should take a look at the migrated file /opt/otobo/Kernel/Config.pm. You may want to adjust custom paths or LDAP settings. In the best case, you will find that some custom settings are no longer needed.

When the migration is complete, please take your time and test the entire system. Once you have decided that the migration was successful and you want to use OTOBO from now on, start the OTOBO daemon:

bash

su - otobo
/opt/otobo/bin/Cron.sh start
/opt/otobo/bin/otobo.Daemon.pl start

In the Docker case:

bash

cd ~/otobo-docker
docker-compose start daemon

Step 6: Clean-Up

Uninstall sshpass if you no longer need it.
Delete the databases and database users created specifically for the migration, if any.
Have fun with OTOBO!

Known Migration Issues

During our migration tests, the browser used for the migration sometimes had problems. After restarting the browser, this problem was usually solved. With Safari, it was sometimes necessary to manually delete the old OTRS session.

Final page of the migration has a strange layout

This can happen if the setting ScriptAlias has a non-standard value. The migration simply replaces otrs with otobo. This can result in the CSS and JavaScript files in OTOBO no longer being retrieved. If this is the case, please check the settings in Kernel/Config.pm and change them back to reasonable values.

Migration stops due to MySQL errors

On systems that have had problems with an upgrade in the past, the migration process can stop due to MySQL errors in the ticket and ticket_history tables. Usually, these are NULL values in the source table that are no longer allowed in the target table. These conflicts must be resolved manually before you can continue the migration.

As of OTOBO 10.0.12, there is a check in migration.pl that checks for NULL values before data transfer. Note that the solution still has to be done manually.

Error in step 5 when migrating to PostgreSQL

In these cases, the not very helpful message "The system could not complete the data transfer." is displayed by migration.pl. The Apache log file and the OTOBO log file show a more meaningful message: " Error message: ERROR: permission denied to set parameter "session_replication_role", SQL: 'set session_replication_role to replica;'" To grant the database user otobo the required superuser privileges, execute the following command as PostgreSQL admin: ALTER USER [otobo](../index.md) WITH SUPERUSER;. Then try to run http://localhost/otobo/migration.pl again. After the migration, revert to the normal state by executing ALTER USER [otobo](../index.md) WITH NOSUPERUSER.

It is not yet clear whether the extended privileges must be granted in every setup.

TIP

The discussion in the OTOBO Forum

Problems with the deployment of the merged system configuration

The system configuration is migrated after the database tables have been migrated. In this context, migration means that the default settings of OTOBO are merged with the system configuration of the source OTRS system. Inconsistencies can occur in this step. A real-world example is the setting Ticket::Frontend::AgentTicketQuickClose###State. This setting is new in OTOBO 10 and the default value is the state closed successful. But this setting is invalid if the state closed successful was deleted or renamed in the source system. This inconsistency is detected as an error in the migration step Migrate configuration settings. The merged system configuration is actually stored in the database, but additional validity checks are performed during deployment.

The problem must be fixed manually using OTOBO console commands.

List the inconsistencies with the command bin/otobo.Console.pl Admin::Config::ListInvalid.
Fix the invalid values interactively with bin/otobo.Console.pl Admin::Config::FixInvalid.
Deploy the collected changes from migration.pl, including the disabled SecureMode, with bin/otobo.Console.pl Maint::Config::Rebuild.

After these manual steps, you should be able to run migration.pl again. The migration will continue with the step where the error occurred.

Final Manual Tasks

Password Policy Rules

With OTOBO 10, a new default password policy for agent and customer users comes into effect when local authentication is used. The password policy rules can be changed in the system configuration (PreferencesGroups###Password and CustomerPersonalPreference###Password).

| Password policy rule | Default |

|-------------------------------------------|--------------------|

| PasswordMinSize | 8 |

| PasswordMin2Lower2UpperCharacters | Yes |

| PasswordNeedDigit | Yes |

| PasswordHistory | 10 |

| PasswordTTL | 30 days |

| PasswordWarnBeforeExpiry | 5 days |

| PasswordChangeAfterFirstLogin | Yes |

Under Docker: Manually migrate cron jobs

In a non-Docker-based installation of OTOBO, there is at least one cron job that checks the health of the daemon. Under Docker, this cron job no longer exists. Furthermore, no cron daemon is running in any of the Docker containers. This means that you have to look for an individual solution for OTRS systems with custom cron jobs (e.g., backup of the database).

Migration from Oracle to Oracle

For migrating to Oracle, the ETL-like strategy must be applied. This is because Oracle does not provide an easy way to temporarily disable foreign key checking.

An Oracle client as well as the Perl module DBD::Oracle must be installed on the OTOBO host.

INFO

When using the Oracle Instant Client, the optional SDK is also required for the installation of DBD:: Oracle.

There are many ways to clone a schema. In the example commands, we use expdp and impdp, which use Data Pump under the hood.

INFO

The connection strings shown in this documentation refer to the case when both the source and target databases are running in a Docker container. See also https://github.com/bschmalhofer/otobo-ideas/blob/master/oracle.md.

Cleanup of otobo

Stop the web server for otobo so that the DB connection for otobo is closed.

SQL

DROP USER otobo CASCADE

Export of the entire OTRS schema.

bash


mkdir /tmp/otrs_dump_dir

SQL


CREATE DIRECTORY OTRS_DUMP_DIR AS '/tmp/otrs_dump_dir';

GRANT READ, WRITE ON DIRECTORY OTRS_DUMP_DIR TO sys;

bash


expdp \"sys/Oradoc_db1@//127.0.0.1/orclpdb1.localdomain as sysdba\" schemas=otrs directory=OTRS_DUMP_DIR dumpfile=otrs.dmp logfile=expdpotrs.log

Import the OTRS schema and rename the schema to 'otobo'.

bash


impdp \"sys/Oradoc_db1@//127.0.0.1/orclpdb1.localdomain as sysdba\" directory=OTRS_DUMP_DIR dumpfile=otrs.dmp logfile=impdpotobo.log remap_schema=otrs:otobo

SQL


select owner, table_name from all_tables where table_name like 'ARTICLE_DATA_OT%_CHAT';
    ALTER USER otobo IDENTIFIED BY XXXXXX;

Adjusting the cloned schema otobo

bash


cd /opt/otobo

scripts/backup.pl --backup-type migratefromotrs # it is okay that the command only knows about the otobo database, only the last line is relevant

sqlplus otobo/otobo@//127.0.0.1/orclpdb1.localdomain < /home/bernhard/devel/OTOBO/otobo/2021-03-31_13-36-55/orclpdb1.localdomain_post.sql >sqlplus.out 2>&1

For checking, use `select owner, table_name from all_tables where table_name like 'ARTICLE_DATA_OT%_CHAT';

Restart the web server for OTOBO
Continue with step 5, i.e., run migration.pl.

INFO

When migrating to an OTOBO version greater than or equal to 10.1, the script /opt/otobo/scripts/DBUpdate-to-10.1.pl must be executed to create the tables stats_report and data_storage newly added in version 10.1.

Optional: Simplified migration of the database (only for experts and special scenarios)

In the general migration strategy, all data in the database tables is copied row by row from the OTRS database to the OTOBO database. Exporting the data from the OTRS database and importing it into the OTOBO database can save time and is more stable in some cases.

INFO

This variant works for both Docker-based and native installations.

INFO

These instructions assume that ((OTRS)) Community Edition uses MySQL as a backend.

First of all, we need a dump of the required OTRS database tables. Then we have to perform some transformations:

Conversion of the character set to utf8mb4
Renaming of some tables
Truncation of some table columns

After the transformation, we can overwrite the tables in the OTOBO schema with the transformed data from OTRS. Effectively, we do not need a single dump file, but several SQL scripts.

If mysqldump is installed and a connection to the OTRS database is possible, you can create the database dump directly on the Docker host. This case is supported by the script bin/backup.pl.

WARNING

This requires that an OTOBO installation is available on the Docker host.

bash

cd /opt/otobo
scripts/backup.pl -t migratefromotrs --db-name otrs --db-host=127.0.0.1 --db-user otrs --db-password "secret_otrs_password"

INFO

Alternatively, the database can be backed up on another server and then transferred to the Docker host. An easy way to do this is to copy /opt/otobo to the server where OTRS is running and execute the same command as above.

The script bin/backup.pl generates four SQL scripts in a dump directory, e.g., in 2021-04-13_12-13-04. To execute the SQL scripts, we need to run the mysql command.

Native installation:

bash

cd <dump_dir>
mysql -u root -p<root_secret> otobo < otrs_pre.sql
mysql -u root -p<root_secret> otobo < otrs_schema_for_otobo.sql
mysql -u root -p<root_secret> otobo < otrs_data.sql
mysql -u root -p<root_secret> otobo < otrs_post.sql

Docker-based installation:

Run the mysql command inside the Docker container db to import the database dump files. Note that the password for the database root is now the password set in the .env file on the Docker host.

bash

cd /opt/otobo-docker
docker-compose exec -T db mysql -u root -p<root_secret> otobo < /opt/otobo/<dump_dir>/otrs_pre.sql
docker-compose exec -T db mysql -u root -p<root_secret> otobo < /opt/otobo/<dump_dir>/otrs_schema_for_otobo.sql
docker-compose exec -T db mysql -u root -p<root_secret> otobo < /opt/otobo/<dump_dir>/otrs_data.sql
docker-compose exec -T db mysql -u root -p<root_secret> otobo < /opt/otobo/<dump_dir>/otrs_post.sql

For a quick check to see if the import worked, you can run the following commands.

bash

mysql -u root -p<root_secret> -e 'SHOW DATABASES'
mysql -u root -p<root_secret> otobo -e 'SHOW TABLES'
mysql -u root -p<root_secret> otobo -e 'SHOW CREATE TABLE ticket'

or when running under Docker

bash

docker-compose exec -T db mysql -u root -p<root_secret> -e 'SHOW DATABASES'
docker-compose exec -T db mysql -u root -p<root_secret> otobo -e 'SHOW TABLES'
docker-compose exec -T db mysql -u root -p<root_secret> otobo -e 'SHOW CREATE TABLE ticket'

The database is now migrated. This means that we can skip the database migration in the next step. Pay attention to the corresponding checkbox.

Migration from ((OTRS)) Community Edition to OTOBO ​

OTOBO Migration Overview ​

Migration Requirements ​

Step by Step Guide ​

Step 1: OTOBO Installation ​

Step 2: Disable SecureMode ​

Step 3: Stop the OTOBO Daemon ​

Optional: sshpass and rsync ​

Step 4: Preparation ​

Stop all relevant services and the OTRS daemon ​

Delete the caches and operational data ​

Optional Step for Docker ​

Copy /opt/otrs to the volume otobo_opt_otobo ​

Step 5: Migrate data ​

Step 6: Clean-Up ​

Known Migration Issues ​

Login not possible after migration ​

Final page of the migration has a strange layout ​

Migration stops due to MySQL errors ​

Error in step 5 when migrating to PostgreSQL ​

Problems with the deployment of the merged system configuration ​

Final Manual Tasks ​

Password Policy Rules ​

Under Docker: Manually migrate cron jobs ​

Migration from Oracle to Oracle ​

Optional: Simplified migration of the database (only for experts and special scenarios) ​

Native installation: ​

Docker-based installation: ​

Migration from ((OTRS)) Community Edition to OTOBO

OTOBO Migration Overview

Migration Requirements

Step by Step Guide

Step 1: OTOBO Installation

Step 2: Disable SecureMode

Step 3: Stop the OTOBO Daemon

Optional: sshpass and rsync

Step 4: Preparation

Stop all relevant services and the OTRS daemon

Delete the caches and operational data

Optional Step for Docker

Copy /opt/otrs to the volume otobo_opt_otobo

Step 5: Migrate data

Step 6: Clean-Up

Known Migration Issues

Login not possible after migration

Final page of the migration has a strange layout

Migration stops due to MySQL errors

Error in step 5 when migrating to PostgreSQL

Problems with the deployment of the merged system configuration

Final Manual Tasks

Password Policy Rules

Under Docker: Manually migrate cron jobs

Migration from Oracle to Oracle

Optional: Simplified migration of the database (only for experts and special scenarios)

Native installation:

Docker-based installation: