Ticket #4622 (closed enhancement: wontfix)

Opened 4 days ago

Last modified 2 days ago

Use parallel zstd by default

Reported by: curtis Owned by:
Priority: minor Milestone:
Component: mc-vfs Version: master
Keywords: compression Cc:
Blocked By: Blocking:
Branch state: no branch Votes for changeset:

Description

Using pzstd speeds up compression considerably on multi-core systems. And incurs only a minor size penalty.

I threw together this patch to try it out. https://github.com/dotysan/mc/commit/589227e13abb99fbe23667203eace3c37b4d12a9

It does pass make check but I haven't hand-tested all scenarios yet.

Anyone have strong thoughts?

Maybe it should be a separate menu item, in case there are really old systems without it? But it's been around since at least 2017.

Related to: #4571

Attachments

0001-Use-all-cores-for-zstd-compression.patch (5.8 KB) - added by curtis 4 days ago.

Change History

Changed 4 days ago by curtis

comment:1 Changed 3 days ago by zaytsev

Relates to #4571.

comment:2 follow-up: ↓ 3 Changed 3 days ago by zaytsev

  • Summary changed from parallel zstd to Use parallel zstd by default

I don't care much either way, because I don't use it. I think the size difference is rather negligible, but I'm not sure whether the users would want all their cores utilized by default, or it's rather the opposite. Also, one could do a fallback to zstd if pzstd is not found with command -v - this is regarding the possible lack of pzstd on some systems.

Andrew, do you have any opinion on that?

comment:3 in reply to: ↑ 2 Changed 3 days ago by andrew_b

Replying to zaytsev:

I don't care much either way, because I don't use it.

Me too.

I think the size difference is rather negligible, but I'm not sure whether the users would want all their cores utilized by default, or it's rather the opposite.

Agree.

Also, one could do a fallback to zstd if pzstd is not found with command -v - this is regarding the possible lack of pzstd on some systems.

Agree again:

$ pzstd
bash: pzstd: command not found

comment:4 Changed 3 days ago by and

Parallel compression will reduce overall compression rate.

Parallel compression has advantages only on BIG files to reduce long-running compress time.

For mc: default should be single stream compress default command.

comment:5 Changed 2 days ago by zaytsev

  • Status changed from new to closed
  • Resolution set to wontfix
  • Milestone Future Releases deleted

Alright, so let's leave it as is. Thanks for your opinions!

Note: See TracTickets for help on using tickets.