Upgrading with Docker (3.7 -> 3.7.1)

1. Overview

This guide provides step-by-step instructions for upgrading an existing Syndeia Cloud containerized deployment to version 3.7.1. For fresh installations, see the Docker Deployment Guide.

Supported upgrade paths:

• Syndeia Cloud 3.7.x (Docker) → 3.7.1 (Docker)

The upgrade uses the setup.py script with the --from flag pointing to the existing installation. The same script handles fresh installs, upgrades, and rollbacks.

The upgrade process handles:

• Secret Preservation: JWT and Play Framework secrets are carried forward from the old installation so active sessions survive the upgrade.

• Configuration Migration: Three migration strategies available — Smart Merge (default), Direct Copy, and New Conf (see Section 4).

• Data Preservation: Cassandra, Kafka, and JanusGraph data directories are untouched — the new containers connect to the same persistent volumes.

• Image Pinning: The new version's image tag is written to .env, ensuring all services use the correct version.

• Rollback Support: The old installation directory is never modified. A single command reverts to the previous version.

2. Prerequisites

2.1 Before You Begin

• Read the Fresh Installation Guide [here] first if you have not previously worked with the Docker deployment. You need to understand the deliverable file structure, .env file, and setup.py.
  
  

• Schedule downtime for any Production service, announce that downtime, and then bring down the Syndeia Docker Compose services gracefully

  cd /opt/icx/syndeia-cloud-3.7-docker sudo docker compose down

   

• Back up your data. While the upgrade preserves all databases, making a backup is your insurance. Don’t proceed without a backup:

  # Back up the deployed directory (location may differ on your site and your last release) sudo tar czf ~/syndeia-backup-$(date +%Y%m%d).tar.gz \ /opt/icx/syndeia-cloud-3.7-docker/ # Back up persistent data (if you bind-mounted your cassandra and kafka data outside of the deployment directory) sudo tar czf ~/syndeia-data-backup-$(date +%Y%m%d).tar.gz \ /var/lib/syndeia-cloud/

   

• Retrieve, Recall, Reuse your secrets. The upgrade script automatically uses secrets from the .env file in the running Syndeia deployment. If you later overrode those secrets, manually, after deploying Syndeia Cloud with the Intercax scripts, then ensure that you, yourself, have the current values. You will need the proper passwords.
  
  

• Verify disk space. The new distribution requires approximately 35 GB: 5 GB for the installation directory and 30 GB for its container images:

  df -h /opt /var/lib # do not attempt an upgrade with inadequate disk space.

   

• Install pyhocon. Smart Merge (the default config migration mode) requires the pyhocon Python library. Without it, the script falls back to Direct Copy mode automatically:

  sudo pip3 install pyhocon # you may tolerate the warning about running pip3 as root: this is needed because it will root that will be running pyhocon

2.2 Software Deliverables

Download the following to a local workstation from the Intercax secure file share (according to locations provided in your license Jira ticket and email):

• syndeia-cloud-docker-3.7.1.zip (A containerized deployment deliverable)

3. Upgrade Procedure

Stage 1: Preparation

Step 1: Transfer Files

Transfer the new deliverable to the target Syndeia server:

From a Linux/macOS workstation:

From a Windows workstation, use WinSCP, FileZilla, or a similar SFTP client.

Step 2: Extract the New Deliverable

1. Log into the target server and navigate to the directory to which you transferred the deliverable.

   ssh <user>@<server-ip> # Navigate to the directory containing the deliverable cd /home/user

2. Unzip the docker deployment deliverable and cd into the extracted folder

   # Unzip the deliverable and cd unzip syndeia-cloud-3.7.1-docker.zip -d syndeia-cloud-3.7.1-docker cd syndeia-cloud-3.7.1-docker

Your current deployment (e.g., /opt/syndeia-cloud-3.7/) remains untouched.

Stage 2: Pull Images

For security and reliability, you must copy the provided Syndeia Cloud Docker images from Intercax's distribution registry into your local docker repository or your organization's private container registry (e.g., Artifactory, Harbor, AWS ECR, Docker Hub, etc.).

For production deployment, we recommend using an internal or cloud hosted private registry. See https://intercax.atlassian.net/wiki/spaces/IN/pages/5365891073/Security+and+Hardening+Containerized+Post-Deployment+Actions#Security-%26-Hardening.

The provided utility, prepare-images.py, automates this process - first authenticate with the Intercax Docker registry

Step 2: Authenticate with Registry

Log in to the Intercax Registry using the docker login command.

Enter the username and password provided in the intercax-docker-registry-credentials.txt file.

If you do not have these credentials, see the Containerized Deployment Guide 3.7.1 Software: Syndeia Cloud (SC) Components section for information on how to obtain these.

Step 3: Pull Container Images

You can pull everything needed all at once, in stages, sequentially, or in parallel. If you are pulling from the Intercax Docker Registry over highspeed links, try this all-at-once, parallel pull. Use the other options here if you need to slow it down or to pull from different registries.

Assuming that worked well, skip to the verification step.

Verify all images are available:

For air-gapped environments, use the approach described in the Docker Deployment Guide, Section 3b.

Stage 3: Upgrade

For HTTP Systems

You do not make a .env file in new syndeia-cloud-<ver>-docker staging area. The necessary environment variables will be retrieved from the prior deployment that you specify with your “from” argument.

For HTTPS Systems

Because it is easier to resolve SSL certificate configuration on stable platforms, do not attempt to upgrade from a 3.7 (0) Docker deployment that was not using HTTPS directly to a 3.7.1 Docker deployment which is using HTTPS. Administrators find it much easier to first configure a stable system for HTTPS and then migrate the application software (e.g. Syndeia) through a version change while keeping the HTTPS configuration unchanged. Contact Intercax if you find you do want help in making a series of recommended changes in one larger all at once session.

When your prior release has already been configured for HTTPS, its runtime .env file and its config directory holds the properties and files (e.g. the certs) that your new release will use.

The necessary environment variables will be retrieved from the prior deployment that you specify with your “from” argument.

One Small Hand Edit

The following change is needed in the HTTPS.conf file prior to running the automated setup process.

Edit the HTTPS.conf of the prior release and change the path to the docker-mounted keystore file as shown

The path no longer has the “conf” folder but the script was written without that awareness. Deliverables from Intercax after 3.7.1 will have this step removed.

Step 4: Run the Upgrade

Run the setup script with the --from flag pointing to the installation you intend to upgrade:

Adjust the path to match your existing installation directory.

The script performs the following automatically:

1. Validates the old installation: Reads .syndeia-version and .env from the old directory.

2. Recovers secrets: Copies PLAY_HTTP_SECRET_KEY, JWT_SHARED_SECRET, CASSANDRA_PASSWORD, and other secrets from the old .env to the new .env.

3. Migrates configuration: Applies the selected migration strategy (see Section 4).

4. Ports artifacts: Copies HTTPS configuration, keystore, and vendor libraries (Teamcenter, DSCR, DSE3) from the old installation.

5. Generates override file: Creates docker-compose.override.yml with the correct UID/GID mappings for the new version.

6. Updates image version: Sets SYNDEIA_VERSION in .env to the new version tag.

7. Stops old containers: Stops containers running from the old installation.

8. Starts new containers: Starts all services from the new installation in dependency order.

9. Health checks: Polls all services until they report healthy.

10. Records upgrade state: Writes .upgrade-state with the path to the old installation (used for rollback).

The upgrade typically takes 3–5 minutes.

Step 5. Move to /opt

At this point, your updated Syndeia service is deployed. This next step is to relocate it to clean up your home directory and to make the distribution available for maintenance and upgrades.

This concludes the installation process. We recommend that you perform the Verification steps at this time. Intercax Support will ask you for the status of the Verification steps when you call to request live assistance.

Additional information on various options follows.

Optional Flags

4. Configuration Migration

The upgrade script offers three configuration-migration strategies.

Smart Merge is the default and is recommended for typical use.

4.1 Smart Merge (Default)

Smart Merge performs a three-way semantic merge of HOCON configuration files:

1. Extracts the old version's default configuration from the deliverable's pristine defaults directory (config/.defaults/).

2. Compares the old defaults against your modified configuration to identify the changes your organization made to tune Syndeia to your purposes.

3. Applies your customer changes onto the new version's defaults.

This preserves both your customizations and any new configuration options added in 3.7.1.

Requirements: Smart Merge requires the pyhocon Python library. If pyhocon is not installed, the script automatically falls back to Direct Copy for all services with a warning.

If the merger is unable to parse the configuration for a specific service, it retries with simpler Direct Copy for that service and logs a warning. If you want us to investigate why your organization’s changes to a properties file led to the file being non-parsed, send us a Support ticket with the log files from the attempt.

4.2 Direct Copy

Copies old configuration files to be used as the new ones. Use this if you have heavily customized configs and want to ensure an exact carry-forward. (Please let us know how you are extensively modifying these configuration files; generally they should be simple.)

Trade-off: Direct Copy does not incorporate new default configuration keys added in 3.7.1. Review the .template reference file (see Section 4.4) after the upgrade to check for new options.

4.3 New Conf

Keeps fresh default configurations for all services. Old configs are saved with a version suffix (e.g., application.conf.3.7.prev) for manual review. Use this when you want full control over which settings to carry forward.

4.4 Change Reference Files

Regardless of which mode is used, the script saves two reference files alongside each changed configuration:

• application.conf.<old_version>.prev — The actual config from your previous installation.

• application.conf.<new_version>.template — Fresh defaults for the new version.

These give administrators the full context of changes, to review what changed.

To compare your now-effective, migrated config against the defaults of the new release:

To compare your now-effective, migrated config against the configuration of the previous release:

4.5 New Services

Services that exist in 3.7.1 but were not present in the old installation (e.g., codebeamer-impl when upgrading from an earlier version) are excluded from configuration migration. They receive fresh defaults only and require no special handling.

4.6 sc-docker-overrides.conf

The shared configuration file sc-docker-overrides.conf always uses the new version on upgrade, regardless of the migration mode. This file contains service discovery URLs that must match the compose file structure. If it differs from the old version, the old file is saved as sc-docker-overrides.conf.<old_version>.prev.

5. Ported Artifacts

The following artifacts are automatically copied from the old installation to the new one:

• HTTPS configuration: config/sc_service_configs/web-gateway/HTTPS.conf and config/keystore/ directory

• Vendor libraries:

  • libs/TC/ (Teamcenter client libraries)

  • libs/dscr/ (Dassault 3DEXPERIENCE Cloud libraries)

  • libs/dse3/ (Dassault 3DEXPERIENCE on-premise libraries)

• License file: license/syndeia.lic (if present in the old installation)

6. Rollback

If the upgrade causes issues, you can revert to the previous version with a single command.

6.1 Automatic Rollback

From the new installation directory:

This:

1. Stops all containers running from the new installation.

2. Reads the old installation path from .upgrade-state.

3. Starts containers from the old installation directory.

4. Runs health checks against the old version.

The new installation directory is left intact for investigation.

6.2 Manual Rollback

If the automatic rollback fails, you can manually revert:

6.3 Data Considerations

• Cassandra: Data is stored in /var/lib/syndeia-cloud/cassandra/. Both old and new containers mount the same directory. Schema changes in 3.7.1 are additive (CREATE IF NOT EXISTS), so rolling back does not cause schema conflicts.

• Kafka: Data is stored in /var/lib/syndeia-cloud/kafka/. Topic data is compatible across minor versions.

• JanusGraph: Data is stored in /var/lib/syndeia-cloud/janusgraph/. Graph data is preserved across versions.

7. Verification

After the upgrade completes, run the following checks.

7.1 Check Version

Verify the running version by checking the image tags:

All Syndeia service images should show the 3.7.1 tag.

7.2 Check All Services

All services should show healthy status.

7.3 Check Core Ports

7.4 Verify Secrets Were Preserved

The output should be empty (no differences), confirming secrets were carried forward.

7.5 Verify Configuration Migration

Spot-check a service config to confirm your customizations were carried forward:

7.6 Login Test

Command line:

Replace <your-password> with your actual superuser password.

Browser:

Open http://<server-ip>:9000. If the web-gateway listens on 127.0.0.1, use an SSH tunnel:

Then open http://localhost:9000. Log in with your existing credentials. Active sessions from before the upgrade should still be valid (secrets are preserved).

7.7 Check Logs

Review logs for any startup errors:

8. Troubleshooting

8.1 Old Installation Not Detected

The --from path must point to a directory containing a valid .env and docker-compose.yml. Verify the path:

If the old installation used a different directory structure, specify the exact path.

8.2 Smart Merge Falls Back to Direct Copy

The script logs a warning indicating which service had a parse error. The fallback is safe — Direct Copy still carries your configs forward. Check the service's config for HOCON syntax issues:

If pyhocon is not installed, all services fall back to Direct Copy. Install it for full smart merge support:

8.3 Secret Recovery Fails

If the old .env file is missing or corrupted, the upgrade will warn and generate new secrets. This invalidates active sessions. To manually set secrets:

Set PLAY_HTTP_SECRET_KEY and JWT_SHARED_SECRET to the values from your backup.

8.4 Service Fails to Start After Upgrade

Check the service log:

Common causes:

• Incompatible configuration: A removed or renamed config key. Compare against the new defaults:

  diff config/sc_service_configs/<service>/application.conf \ config/sc_service_configs/<service>/application.conf.3.7.1.template

• Image not found: Verify the image was pulled:

  docker images | grep <service-name>

8.5 Health Check Timeout

Some services take longer to start on first boot after upgrade (e.g., Cassandra schema migration). Re-run the health check:

If services are still starting, wait and check again. To increase the timeout:

8.6 Rollback Fails

If setup.py --rollback fails, use the manual rollback procedure in Section 6.2. Verify the old compose file is intact:

8.7 Re-running After Failure

The upgrade script is idempotent. Fix the reported issue and re-run:

No cleanup is needed between attempts.

9. Appendix: What Happens During an Upgrade

This section describes the internal upgrade sequence for reference. No action is required — the script handles all steps automatically.

1. Pre-flight validation: Checks Python version, Docker/Compose availability, .env and docker-compose.yml presence in both old and new directories.

2. Old installation detection: Reads .syndeia-version from the old directory to determine the source version.

3. Secret recovery: Extracts PLAY_HTTP_SECRET_KEY, JWT_SHARED_SECRET, CASSANDRA_PASSWORD, and other secrets from the old .env file.

4. Configuration migration: Applies the selected migration strategy (Smart Merge, Direct Copy, or New Conf). Saves reference files (.prev and .template) for all changed configs.

5. Artifact porting: Copies HTTPS config, keystore, vendor libraries, and license file from the old installation.

6. Override generation: Creates docker-compose.override.yml with user: directives matching the system's syndeia-cloud UID and appropriate GIDs for infrastructure containers.

7. Image version pinning: Updates SYNDEIA_VERSION in .env to the new version tag.

8. Shared config regeneration: Regenerates sc-docker-overrides.conf with service discovery URLs for the new compose structure.

9. Container stop: Stops all containers from the old installation (docker compose down in the old directory).

10. Container start: Starts infrastructure containers, waits for health checks, then starts application services in dependency order.

11. Health checks: Polls docker compose ps until all services report healthy or timeout.

12. State recording: Writes .upgrade-state with the old installation path for rollback support.