feat(volume-backups): add gzip compression to volume backups by anxo-outeiral · Pull Request #5206 · Dokploy/dokploy

anxo-outeiral · 2026-08-28T15:57:17Z

What is this PR about?

Volume backups currently create uncompressed .tar archives using tar cvf. This means that for data that compresses well (SQLite databases, JSON, text logs, configuration files), the backup stored in S3 is the same size as the original volume — wasting storage space and increasing S3 costs.

This PR changes to tar czf (gzip compression), producing .tar.gz files that are significantly smaller for compressible data, reducing S3 storage costs and transfer time.

Changes

backup.ts: Changed tar cvf → tar czf and extension .tar → .tar.gz
utils.ts: Updated retention filter --include "*.tar" → --include "*.tar.gz" and error cleanup *.tar → *.tar.gz

Compatibility

gzip is pre-installed in the ubuntu Docker image used for volume backups — no additional dependencies or runtime package installation needed
Restore is backward compatible: tar xvf in restore.ts auto-detects gzip compression, so existing restore logic continues to work for both old .tar and new .tar.gz files without any changes
Old .tar files already in S3 are not affected by the new retention policy (which now filters *.tar.gz); they will need manual cleanup

Testing

Tested locally with a full development stack (Dokploy dev server + MinIO as local S3 destination + Docker Swarm):

Verified gzip is available in the ubuntu Docker image
Created volume backup with gzip compression
Verified .tar.gz file appears in S3 destination (MinIO)
Verified file command reports gzip compressed data, from Unix
Restored from .tar.gz backup successfully
Verified keepLatestCount retention works with .tar.gz extension
Verified backward compatibility: tar xvf restores both .tar and .tar.gz

Compression results (13.4MB of text/JSON/SQL data):

	Size
Volume (uncompressed)	13.4 MB
Backup in S3 (.tar.gz)	884 KB
Compression ratio	~15:1

Checklist

Before submitting this PR, please make sure that:

You created a dedicated branch based on the canary branch.
You have read the suggestions in the CONTRIBUTING.md file https://github.com/Dokploy/dokploy/blob/canary/CONTRIBUTING.md#pull-request
You have tested this PR in your local instance. If you have not tested it yet, please do so before submitting. This helps avoid wasting maintainers time reviewing code that has not been verified by you.

Known limitations

Overlapping backup schedules for the same service (appName) share a directory, and the error cleanup uses a broad *.tar* glob that may delete another run's in-progress archive. A follow-up should clean up only the specific archive created by the failing run.

Issues related (if applicable)

Closes #3566

Screenshots (if applicable)

Backup log showing tar czf and .tar.gz:

Backup file name: gzip-test-vol-2026-08-28T15-31-31-553Z.tar.gz
Turning off volume backup: Yes
Starting volume backup
Volume backup done ✅
Starting upload to S3...
Upload to S3 done ✅
Local backup file cleaned up ✅

Confidence Score: 4/5

The PR is not yet safe to merge because failure cleanup can delete a concurrent backup run's in-progress archive.

Summary

Changes volume archive creation from uncompressed tar to gzip-compressed tar.
Restricts remote retention matching to .tar.gz files.
Expands local failure cleanup to cover both current and legacy archive extensions.

_{Reviews (3) · Last reviewed commit: "fix(volume-backups): clean up legacy .ta..."}

greptile-apps · 2026-08-28T15:59:09Z

+		// delete all the .tar.gz files
+		const command = `rm -rf ${volumeBackupPath}/*.tar.gz`;


Legacy archives escape cleanup

If an interrupted pre-upgrade backup leaves a local .tar archive, the changed failure handler matches only .tar.gz, leaving the legacy archive on disk indefinitely and consuming server storage.

Knowledge Base Used: Backups and restore

Volume backups were creating uncompressed .tar archives using tar cvf. This changes to tar czf (gzip compression), producing .tar.gz files that are significantly smaller for compressible data (text, JSON, SQLite, logs), reducing S3 storage costs. Changes: - backup.ts: tar cvf -> tar czf, extension .tar -> .tar.gz - utils.ts: retention filter *.tar -> *.tar.gz, error cleanup updated Compatibility: - gzip is pre-installed in the ubuntu image used for volume backups - tar xvf in restore auto-detects gzip, so existing restore logic works for both old .tar and new .tar.gz files - Old .tar files in S3 are not affected by the new retention policy (which now filters *.tar.gz); they require manual cleanup Closes Dokploy#3566

narcisonunez · 2026-09-10T22:57:42Z

failed backup cleanup still leaves legacy local .tar archives behind

greptile-apps · 2026-09-11T12:00:20Z

-		// delete all the .tar files
-		const command = `rm -rf ${volumeBackupPath}/*.tar`;
+		// delete all the .tar and .tar.gz files (including legacy .tar from pre-upgrade)
+		const command = `rm -rf ${volumeBackupPath}/*.tar*`;


Cleanup deletes concurrent archives

If two runs of the same volume-backup record overlap, a failure in one run deletes every *.tar* file in their shared directory, including the other run's in-progress .tar.gz archive, causing its tar creation or upload to fail as well.

Knowledge Base Used: Backups and restore

@greptile-apps it's a valid concern. I think this was already discussed in the original fix PR (#2652). Here's the context:

Each volume backup job writes to its own directory: ${VOLUME_BACKUPS_PATH}/${volumeBackup.appName}/ — so jobs for different backup records are isolated. When @Siumauricio reviewed the original fix he confirmed: "when you create a new volume backup job it creates its own directory, so they are isolated."

That said, two schedules targeting the same service (same appName) would share a directory and could theoretically collide if they overlap. That's a known limitation of the locking mechanism and out of scope for this PR, but worth addressing in a follow-up (e.g. by using the in-progress backup filename as the cleanup target instead of a broad glob).

Should I add a note about this in the PR description as a known limitation, or do you think it's worth blocking on?

That context is valid. Given that each backup record normally has its own ${VOLUME_BACKUPS_PATH}/${volumeBackup.appName} directory, I would not block this PR solely on the concurrency concern—especially since overlapping schedules for the same service are a pre-existing limitation rather than something introduced by gzip.

I would add a short “Known limitation” note to the PR description and track the follow-up. Be precise: overlapping backups that resolve to the same appName can share the directory, and failure cleanup currently uses a broad *.tar* glob that may remove another run’s archive. The follow-up should clean up only the archive created by the failing run, ideally by passing the exact backup path into the failure cleanup.

Recommendation: downgrade this from a merge blocker to a non-blocking follow-up, provided per-record directory isolation is confirmed for all supported service types. The broader glob is still worth fixing separately because it makes that existing race affect the new .tar.gz files.

Tip: You can customize Greptile's behavior for this repo with .greptile/rules.md and .greptile/config.json.

@greptile-apps confirmed! Directory isolation (${VOLUME_BACKUPS_PATH}/${volumeBackup.appName}/) applies consistently across all service types: application, compose, postgres, mysql, mariadb, mongo, redis, and libsql. The broader cleanup glob is a pre-existing issue and worth addressing separately in a follow-up.

I've also added a "Known limitations" section to the PR description to document this

Confirmed. Since directory isolation applies consistently across all supported service types, this is not a merge blocker for this PR. Documenting the shared-appName concurrency case as a known limitation is appropriate; the broad cleanup glob can be addressed separately by targeting the failing run’s exact archive path.

anxo-outeiral requested a review from Siumauricio as a code owner August 28, 2026 15:57

dosubot Bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Aug 28, 2026

greptile-apps Bot reviewed Aug 28, 2026

View reviewed changes

anxo-outeiral mentioned this pull request Aug 28, 2026

Supporting compression on volume backups #3566

Open

anxo-outeiral force-pushed the feat/volume-backup-gzip-compression branch from d01baef to 76615ad Compare August 28, 2026 16:23

[autofix.ci] apply automated fixes

02e405d

fix(volume-backups): clean up legacy .tar files on backup failure

eb8fde3

greptile-apps Bot reviewed Sep 11, 2026

View reviewed changes

hamidzr mentioned this pull request Nov 5, 2026

Show backup storage usage: totals per schedule and destination, plus sizes for volume backups #5884

Open

Navigation Menu

Repository navigation

feat(volume-backups): add gzip compression to volume backups - #5206

feat(volume-backups): add gzip compression to volume backups#5206
anxo-outeiral wants to merge 3 commits into
Dokploy:canaryfrom
anxo-outeiral:feat/volume-backup-gzip-compression

anxo-outeiral commented Aug 28, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot Aug 28, 2026

Uh oh!

narcisonunez commented Sep 10, 2026

Uh oh!

greptile-apps Bot Sep 11, 2026

Uh oh!

anxo-outeiral Sep 11, 2026

Uh oh!

greptile-apps Bot Sep 11, 2026

Uh oh!

anxo-outeiral Sep 11, 2026

Uh oh!

greptile-apps Bot Sep 11, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		// delete all the .tar.gz files
		const command = `rm -rf ${volumeBackupPath}/*.tar.gz`;

Conversation

anxo-outeiral commented Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What is this PR about?

Changes

Compatibility

Testing

Checklist

Known limitations

Issues related (if applicable)

Screenshots (if applicable)

Confidence Score: 4/5

Summary

Uh oh!

greptile-apps Bot Aug 28, 2026

Choose a reason for hiding this comment

Uh oh!

narcisonunez commented Sep 10, 2026

Uh oh!

greptile-apps Bot Sep 11, 2026

Choose a reason for hiding this comment

Uh oh!

anxo-outeiral Sep 11, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Sep 11, 2026

Choose a reason for hiding this comment

Uh oh!

anxo-outeiral Sep 11, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Sep 11, 2026

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

anxo-outeiral commented Aug 28, 2026 •

edited

Loading