Migration from ((OTRS)) Community Edition to OTOBO

Welcome and thank you for choosing OTOBO! OTRS, ((OTRS)) Community Edition, and OTOBO are very comprehensive and flexible in their application. Therefore, every migration to OTOBO requires thorough preparation and possibly some follow-up work.

Please take your time for the migration and follow these instructions step by step.

::: info

After the migration, the data previously available in ((OTRS)) Community Edition 6 is available in OTOBO 10. We do not change any data of the OTRS 6 installation during the migration.

:::

OTOBO Migration Overview

With the OTOBO migration interface, it is possible to use the following migration strategies:

The general migration strategy.

This is the regular way to perform a migration. Many different combinations are supported:
- Server change: Migrate and switch to a new application server at the same time.
- Separate application and web servers: You have the choice of whether to run application and database servers on the same host or on a dedicated host each. This choice is independent of the previous setup in OTRS / ((OTRS)) Community Edition.
- Different databases: Migrate from any supported database to another supported database.
- Different operating systems: Switch from any supported operating system to another supported operating system.
- Docker: Migrate to a Docker-based installation of OTOBO 10.
A variant of the general strategy where the database migration is optimized.

Use the ETL-like migration if the source database must not be burdened by increased load or if access to the source database is a bottleneck. In the general strategy, the data is read row by row from the otrs database first and then inserted into the OTOBO database. In this variant, the complete otrs database tables are exported first, then transformed, and then imported into the otobo database.

Migration from an Oracle-based ((OTRS)) Community Edition 6 installation to an Oracle-based OTOBO installation.

This is a special case that is not supported by the general migration strategy. This means that a variant of the optimized strategy must be used.

::: warning

All strategies work for both Docker-based and native installations. However, for Docker-based installations, some special features must be taken into account. These special features are covered in the optional steps.

:::

::: info

It is also feasible to clone the ((OTRS)) Community Edition database to the OTOBO database server before the actual migration. This can speed up the general migration strategy.

:::

Migration Requirements

The basic requirement for a migration is that you already have an ((OTRS)) Community Edition or OTRS 6.0.* running and want to transfer both configuration and data to OTOBO.

::: warning

Please consider carefully whether you really need the data and configuration. Experience shows that in many cases, a fresh start is the better option. This is because the previously used installation and configuration was often rather suboptimal. It might also make sense to transfer only the ticket data and switch the basic configuration to OTOBO best practices.

:::
You need a running OTOBO installation to start the migration from there!
This OTOBO installation must contain all OPM packages that are installed in your ((OTRS)) Community Edition and that you also want to use in OTOBO.
If you plan to migrate to a different server, the OTOBO web server must be able to access the location where your ((OTRS)) Community Edition or OTRS 6.0.* is installed. In most cases, this is the /opt/otrs* directory on the server where ((OTRS)) Community Edition is running. Read access can be provided via SSH or via file system mounts.
The ((otrs)) Community Edition database must be accessible from the server where OTOBO is running. Read-only access must be granted for external hosts. If access is not possible or if the speed of the migration is to be optimized, then a dump of the database is sufficient.

::: info

If SSH and database access between the servers are not possible, please migrate ((OTRS)) Community Edition to OTOBO on the same server and then move the new installation.

:::

Step-by-Step Guide

Step 1: OTOBO Installation

Please start with the installation of a new OTOBO system. Your old OTRS / ((OTRS)) Community Edition installation will be migrated to this new system.

::: warning

Under Apache, there are pitfalls when running two independent mod_perl applications on the same web server. Therefore, it is recommended to run ((OTRS)) Community Edition and OTOBO on separate web servers. Alternatively, the OTRS configuration should be removed from Apache before OTOBO is installed. Use the command a2query -s and check the directories /etc/apache2/sites-available and /etc/apache2/sites-enabled to see which configurations are currently available and enabled.

:::

After completing the installation, please log in as root@localhost. Navigate to the OTOBO Admin area Admin -> Package Manager and install all required OTOBO OPM packages.

The following OPM packages and ((OTRS)) Community Edition “Feature Addons” do NOT need to be and should NOT be installed, as these functions are already included in the OTOBO standard:

OTRSHideShowDynamicField
RotherOSSHideShowDynamicField
TicketForms
RotherOSS-LongEscalationPerformanceBoost
Znuny4OTRS-AdvancedDynamicFields
Znuny4OTRS-AutoSelect
Znuny4OTRS-EscalationSuspend
OTRSEscalationSuspend
OTRSDynamicFieldDatabase
OTRSDynamicFieldWebService
OTRSBruteForceAttackProtection
Znuny4OTRS-ExternalURLJump
Znuny4OTRS-QuickClose
Znuny4OTRS-AutoCheckbox
OTRSSystemConfigurationHistory
Znuny4OTRS-PasswordPolicy

The following OTOBO packages have been integrated into OTOBO 10+. This means that they should not be installed on the target system if the target system is OTOBO 10+. - ImportExport

Step 2: Disable SecureMode

After you have installed OTOBO, please log in to the OTOBO Admin area again Admin -> System Configuration and disable the configuration option SecureMode.

::: info

Do not forget to actually implement the changed setting.

:::

Step 3: Stop OTOBO Daemon

This is necessary if the OTOBO daemon is actually running. Stopping the daemon differs between Docker-based and non-Docker-based installations.

In the non-Docker case, execute the following commands as user otobo:

# if you are logged in as root

root> su - otobo

otobo> /opt/otobo/bin/Cron.sh stop

otobo> /opt/otobo/bin/otobo.Daemon.pl stop --force

If OTOBO is running in Docker, you only need to stop the daemon service:

docker_admin> cd /opt/otobo-docker

docker_admin> docker-compose stop daemon

docker_admin> docker-compose ps     # otobo_daemon_1 should have exited with code 0

:::: info

It is recommended to perform a backup of the entire OTOBO system at this point. If something goes wrong during the migration, you do not have to repeat the entire installation process, but can instead import the backup for a new migration.

::: tip

We recommend that you read the chapter OTOBO Backup backup-restore.

:::

::: details Optional: Mount /opt/otrs

Often, OTOBO is intended to run on a new server where /opt/otrs is not initially available. In these cases, the /opt/otrs directory on the ((OTRS)) Community Edition server can be mounted into the file system of the OTOBO server. If a regular network mount is not possible, sshfs could be an option.

Optional: sshpass and rsync

This step is only necessary if you want to migrate ((OTRS)) Community Edition from another server and /opt/otrs has not been mounted from the remote server onto the OTOBO server.

The tools sshpass and rsync are required so that migration.pl can copy files via ssh. To install sshpass, please log in as user root on the server and execute one of the following commands:

::: details Install sshpass on Debian / Ubuntu Linux

sudo apt install sshpass

:::

::: details Install sshpass on RHEL/CentOS Linux

sudo yum install sshpass

:::

::: details Install sshpass on Fedora Linux

sudo dnf install sshpass

:::

::: details Install sshpass on OpenSUSE Linux

sudo zypper install sshpass

:::

The same must be done for rsync if it is not already available.

Step 4: Preparation

::: info

Make sure that you also have a valid backup of your OTRS / ((OTRS)) Community Edition system. Yes, we do not touch any ((OTRS)) Community Edition data during the migration, but sometimes a wrong entry is enough to cause problems. :::

Now we are ready for the migration. First, we must ensure that no more tickets are being processed and no users are logging in to ((OTRS)) Community Edition:

Please log in to the ((OTRS)) Community Edition Admin area Admin -> System Maintenance and add a new system maintenance slot for a few hours. Afterwards, delete all agent and user sessions (Admin -> Sessions) and log out.

Stop all relevant services and the OTRS daemon

Please ensure that no services or cron jobs are running.

su - otrs
/opt/otrs/bin/Cron.sh stop
/opt/otrs/bin/otrs.Daemon.pl stop --force

Delete caches and operational data

The cached data and operational data do not need to be migrated. The email queue should already be empty at this point.

su - otrs
/opt/otrs/bin/otrs.Console.pl Maint::Cache::Delete
/opt/otrs/bin/otrs.Console.pl Maint::Session::DeleteAll
/opt/otrs/bin/otrs.Console.pl Maint::Loader::CacheCleanup
/opt/otrs/bin/otrs.Console.pl Maint::WebUploadCache::Cleanup
/opt/otrs/bin/otrs.Console.pl Maint::Email::MailQueue --delete-all

Optional step for Docker

There are some special features to consider if your OTOBO installation is running under Docker. The most relevant: processes running in a Docker container generally cannot access directories outside the container. However, there is an exception: directories that are mounted as volumes into the container can be accessed. Also note that the MariaDB database running in otobo_db_1 (otobo-db-1 in newer docker compose versions) is not directly accessible from outside the container network.

::: info

In the example commands, we assume that the user docker_admin is used for interaction with Docker. The Docker administrator can be either the root user of the Docker host or a dedicated user with the required permissions.

:::

Copy /opt/otrs into the volume otobo_opt_otobo

::: note Container Names In old docker compose versions, the container names are otobo_web_1, otobo_db_1, and otobo_daemon_1. In newer versions, the container names are otobo-web-1, otobo-db-1, and otobo-daemon-1.

:::

In this section, we assume that the OTRS home directory /opt/otrs is available on the Docker host.

There are at least two viable options:

a. Copy /opt/otrs into the existing volume otobo_opt_otobo

b. Mount /opt/otrs as an additional volume

Let’s focus here on option a..

First, we need to find out where the volume otobo_opt_otobo is available on the Docker host.

 otobo_opt_otobo_mp=$(docker volume inspect --format '{{ .Mountpoint }}' otobo_opt_otobo)
echo $otobo_opt_otobo_mp

For secure copying, we use rsync. Depending on your Docker configuration, the rsync command may need to be executed with sudo.

rsync --recursive --safe-links --owner --group --chown 1000:1000 --perms --chmod "a-wx,Fu+r,Du+rx" /opt/otrs/ $otobo_opt_otobo_mp/var/tmp/copied_otrs
ls -la $otobo_opt_otobo_mp/var/tmp/copied_otrs  # for verification only
sudo rsync --recursive --safe-links --owner --group --chown 1000:1000 --perms --chmod "a-wx,Fu+r,Du+rx" /opt/otrs/ $otobo_opt_otobo_mp/var/tmp/copied_otrs
sudo ls -la $otobo_opt_otobo_mp/var/tmp/copied_otrs  # for verification only

This copied directory will be available inside the container as /opt/otobo/var/tmp/copied_otrs.

Step 5: Migrate Data

:::note Docker Compose Command In old Docker Compose versions, the command docker-compose was used. In newer versions, docker compose is used. :::

Please use the web migration tool at http://localhost/otobo/migration.pl. Note that you may need to replace “localhost” with your OTOBO hostname and possibly add your non-standard port. The application will then guide you through the migration process.

::: warning

Sometimes a warning is displayed that the deactivation of SecureMode was not detected. In this case, please restart the web server. This forces the web server to read the current configuration.

service apache2 restart
cd /opt/otobo-docker
docker-compose restart web
docker-compose ps

:::

::: info

If OTOBO is running in a Docker container, keep the default settings localhost for the OTRS server and /opt/otobo/var/tmp/copied_otrs for the OTRS home directory. This is the path of the data that was copied in the optional step.

:::

::: info

The default values for OTRS database user and password are taken from Kernel/Config.pm in the OTRS home directory. Change the suggested settings if you are using a dedicated database user for the migration. Also change the settings if you are working with a database that was copied into the otobo_db_1 Docker container.

:::

::: info

In the Docker case, a database running on the Docker host is not reachable via 127.0.0.1 inside the Docker container. This means that the setting 127.0.0.1 will not be valid for the OTRS-Server input field. In this case, enter one of the alternative IP addresses reported by the command hostname --all-ip-addresses for OTRS-Server.

:::

::: info

When migrating to a new application server or to a Docker-based installation, the database often cannot be accessed from the target installation. This is usually because the otobo database user can only connect from the host where the database is running. To still allow access, it is recommended to create a dedicated database user for the migration, e.g., CREATE [USER](../agents/agents.md) 'otrs_migration'@'%' IDENTIFIED BY 'otrs_migration'; and GRANT SELECT, SHOW VIEW ON otrs.* TO 'otrs_migration'@'%';. This user can be deleted again after the migration: DROP [USER](../agents/agents.md) 'otrs_migration'@'%'.

:::

User-specific settings in Kernel/Config.pm are transferred from the old OTRS installation to the new OTOBO installation. If you have user-specific settings, you should take a look at the migrated file /opt/otobo/Kernel/Config.pm. You might want to adjust custom paths or LDAP settings. In the best case, you will find that some custom settings are no longer needed.

When the migration is complete, please take your time and test the entire system. Once you have decided that the migration was successful and you want to use OTOBO from now on, start the OTOBO daemon:

su - otobo
/opt/otobo/bin/Cron.sh start
/opt/otobo/bin/otobo.Daemon.pl start

In the Docker case:

cd ~/otobo-docker
docker-compose start daemon

Step 6: Clean-Up

Uninstall sshpass if you no longer need it.
Delete the databases and database users that were created specifically for the migration, if any.
Have fun with OTOBO!

Known Migration Issues

During our migration tests, the browser used for the migration sometimes had problems. After restarting the browser, this problem was usually solved. With Safari, it was sometimes necessary to manually delete the old OTRS session.

Final page of the migration has a strange layout

This can happen if the ScriptAlias setting has a non-standard value. The migration simply replaces otrs with otobo. This can lead to the CSS and JavaScript files in OTOBO no longer being retrievable. If this is the case, please check the settings in Kernel/Config.pm and change them back to reasonable values.

Migration stops due to MySQL errors

On systems that have had problems with an upgrade in the past, the migration process may stop due to MySQL errors in the ticket and ticket_history tables. These are usually NULL values in the source table that are no longer permitted in the target table. These conflicts must be resolved manually before you can continue the migration.

As of OTOBO 10.0.12, there is a check in migration.pl that checks for NULL values before data transfer. Note that the solution must still be performed manually.

Error in step 5 during migration to PostgreSQL

In these cases, the not very helpful message “The system could not complete the data transfer.” is displayed by migration.pl. The Apache log file and the OTOBO log file show a more meaningful message: “Error message: ERROR: permission denied to set parameter “session_replication_role”, SQL: ‘set session_replication_role to replica;’” To grant the database user otobo the required superuser privileges, execute the following command as PostgreSQL admin: ALTER USER [otobo](../index.md) WITH SUPERUSER;. Then try to execute http://localhost/otobo/migration.pl again. After the migration, return to the normal state by executing ALTER USER [otobo](../index.md) WITH NOSUPERUSER.

It is not yet clear whether the extended privileges must be granted in every setup.

::: tip

The discussion in the OTOBO Forum

:::

Problems with the deployment of the merged system configuration

The system configuration is migrated after the database tables have been migrated. In this context, migration means that the default settings of OTOBO are merged with the system configuration of the source OTRS system. In this step, inconsistencies can occur. A real-world example is the setting Ticket::Frontend::AgentTicketQuickClose###State. This setting is new in OTOBO 10 and the default value is the state closed successful. But this setting is invalid if the state closed successful was deleted or renamed in the source system. This inconsistency is detected as an error in the migration step Migrate configuration settings. The merged system configuration is actually saved in the database, but additional validity checks are performed during deployment.

The problem must be resolved manually using OTOBO console commands.

List the inconsistencies with the command bin/otobo.Console.pl Admin::Config::ListInvalid.
Fix the invalid values interactively with bin/otobo.Console.pl Admin::Config::FixInvalid.
Deploy the collected changes from migration.pl, including the deactivated SecureMode, with bin/otobo.Console.pl Maint::Config::Rebuild.

After these manual steps, you should be able to execute migration.pl again. The migration will continue from the step where the error occurred.

Final Manual Tasks

Password Policy Rules

With OTOBO 10, a new default password policy for agents and customer users comes into effect when local authentication is used. The password policy rules can be changed in the system configuration (PreferencesGroups###Password and CustomerPersonalPreference###Password).

Password Policy Rule	Default
`PasswordMinSize`	8
`PasswordMin2Lower2UpperCharacters`	Yes
`PasswordNeedDigit`	Yes
`PasswordHistory`	10
`PasswordTTL`	30 days
`PasswordWarnBeforeExpiry`	5 days
`PasswordChangeAfterFirstLogin`	Yes

Under Docker: Manually migrate cron jobs

In a non-Docker-based installation of OTOBO, there is at least one cron job that checks the health of the daemon. Under Docker, this cron job no longer exists. Furthermore, no cron daemon runs in any of the Docker containers. This means that you have to look for an individual solution for OTRS systems with user-specific cron jobs (e.g., backup of the database).

Migration from Oracle to Oracle

For the migration to Oracle, the ETL-like strategy must be applied. This is because Oracle does not offer an easy way to temporarily turn off foreign key checking.

On the OTOBO host, an Oracle client as well as the Perl module DBD::Oracle must be installed.

::: info

When using the Oracle Instant Client, the optional SDK is also required for the installation of DBD::Oracle.

:::

There are many ways to clone a schema. In the example commands, we use expdp and impdp, which use Data Pump under the hood.

::: info

The connection strings shown in this documentation refer to the case when both the source and target databases are running in a Docker container. See also https://github.com/bschmalhofer/otobo-ideas/blob/master/oracle.md.

:::

Cleanup of otobo

Stop the web server for otobo so that the DB connection for otobo is closed.

DROP USER otobo CASCADE

Export of the entire OTRS schema.

mkdir /tmp/otrs_dump_dir

CREATE DIRECTORY OTRS_DUMP_DIR AS '/tmp/otrs_dump_dir';
GRANT READ, WRITE ON DIRECTORY OTRS_DUMP_DIR TO sys;

expdp \"sys/Oradoc_db1@//127.0.0.1/orclpdb1.localdomain as sysdba\" schemas=otrs directory=OTRS_DUMP_DIR dumpfile=otrs.dmp logfile=expdpotrs.log

Import the OTRS schema and rename the schema to ‘otobo’.

impdp \"sys/Oradoc_db1@//127.0.0.1/orclpdb1.localdomain as sysdba\" directory=OTRS_DUMP_DIR dumpfile=otrs.dmp logfile=impdpotobo.log remap_schema=otrs:otobo

select owner, table_name from all_tables where table_name like 'ARTICLE_DATA_OT%_CHAT';
ALTER USER otobo IDENTIFIED BY XXXXXX;

Adjust the cloned schema otobo

cd /opt/otobo
scripts/backup.pl --backup-type migratefromotrs # it is okay that the command only knows about the otobo database, only the last line is relevant

sqlplus otobo/otobo@//127.0.0.1/orclpdb1.localdomain < /home/bernhard/devel/OTOBO/otobo/2021-03-31_13-36-55/orclpdb1.localdomain_post.sql >sqlplus.out 2>&1

For verification, use select owner, table_name from all_tables where table_name like 'ARTICLE_DATA_OT%_CHAT';

Restart the web server for OTOBO
Continue with step 5, i.e., execute migration.pl.

::: info

When migrating to an OTOBO version greater than or equal to 10.1, the script /opt/otobo/scripts/DBUpdate-to-10.1.pl must be executed to create the tables stats_report and data_storage newly added in version 10.1.

:::

Optional: Simplified migration of the database (only for experts and special scenarios)

In the general migration strategy, all data in the database tables is copied row by row from the OTRS database to the OTOBO database. Exporting the data from the OTRS database and importing it into the OTOBO database can save time and is more stable in some cases.

::: info

This variant works for both Docker-based and native installations.

:::

::: info

These instructions assume that ((OTRS)) Community Edition uses MySQL as a backend.

:::

First of all, we need a dump of the required OTRS database tables. Then we have to perform some transformations:

Conversion of the character set to utf8mb4
Renaming of some tables
Truncation of some table columns

After the transformation, we can overwrite the tables in the OTOBO schema with the transformed data from OTRS. Effectively, we do not need a single dump file, but several SQL scripts.

If mysqldump is installed and a connection to the OTRS database is possible, you can create the database dump directly on the Docker host. This case is supported by the script bin/backup.pl.

::: warning

This requires that an OTOBO installation is available on the Docker host.

:::

cd /opt/otobo
scripts/backup.pl -t migratefromotrs --db-name otrs --db-host=127.0.0.1 --db-user otrs --db-password "secret_otrs_password"

::: info

Alternatively, the database can be backed up on another server and then transferred to the Docker host. An easy way to do this is to copy /opt/otobo to the server where OTRS is running and execute the same command as above.

:::

The script bin/backup.pl generates four SQL scripts in a dump directory, e.g., in 2021-04-13_12-13-04. To execute the SQL scripts, we have to execute the mysql command.

Native Installation:

cd <dump_dir>
mysql -u root -p<root_secret> otobo < otrs_pre.sql
mysql -u root -p<root_secret> otobo < otrs_schema_for_otobo.sql
mysql -u root -p<root_secret> otobo < otrs_data.sql
mysql -u root -p<root_secret> otobo < otrs_post.sql

Docker-based Installation:

Execute the mysql command inside the Docker container db to import the database dump files. Note that the password for the database root is now the password that was set in the .env file on the Docker host.

cd /opt/otobo-docker
docker-compose exec -T db mysql -u root -p<root_secret> otobo < /opt/otobo/<dump_dir>/otrs_pre.sql
docker-compose exec -T db mysql -u root -p<root_secret> otobo < /opt/otobo/<dump_dir>/otrs_schema_for_otobo.sql
docker-compose exec -T db mysql -u root -p<root_secret> otobo < /opt/otobo/<dump_dir>/otrs_data.sql
docker-compose exec -T db mysql -u root -p<root_secret> otobo < /opt/otobo/<dump_dir>/otrs_post.sql

For a quick check if the import worked, you can execute the following commands.

mysql -u root -p<root_secret> -e 'SHOW DATABASES'
mysql -u root -p<root_secret> otobo -e 'SHOW TABLES'
mysql -u root -p<root_secret> otobo -e 'SHOW CREATE TABLE ticket'

or when running under Docker

docker-compose exec -T db mysql -u root -p<root_secret> -e 'SHOW DATABASES'
docker-compose exec -T db mysql -u root -p<root_secret> otobo -e 'SHOW TABLES'
docker-compose exec -T db mysql -u root -p<root_secret> otobo -e 'SHOW CREATE TABLE ticket'

The database is now migrated. This means that we can skip the database migration in the next step. Pay attention to the corresponding checkbox.

Migration from ((OTRS)) Community Edition to OTOBO

Migration from ((OTRS)) Community Edition to OTOBO

OTOBO Migration Overview

Migration Requirements

Step-by-Step Guide

Step 1: OTOBO Installation

Step 2: Disable SecureMode

Step 3: Stop OTOBO Daemon

Optional: sshpass and rsync

Step 4: Preparation

Stop all relevant services and the OTRS daemon

Delete caches and operational data

Optional step for Docker

Copy /opt/otrs into the volume otobo_opt_otobo

Step 5: Migrate Data

Step 6: Clean-Up

Known Migration Issues

Login not possible after migration

Final page of the migration has a strange layout

Migration stops due to MySQL errors

Error in step 5 during migration to PostgreSQL

Problems with the deployment of the merged system configuration

Final Manual Tasks

Password Policy Rules

Under Docker: Manually migrate cron jobs

Migration from Oracle to Oracle

Optional: Simplified migration of the database (only for experts and special scenarios)

Native Installation:

Docker-based Installation: