How to limit size of appendonly.aof redis file

Question

How to limit appendonly.aof file ?

My appendonly file grow to 92g.

It's for the stack gitlab, so for sessions,...

Here the info and auto_aof_rewrite:

127.0.0.1:6379> info
# Server
redis_version:6.0.9
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:5f0299f43563a32b
redis_mode:standalone
os:Linux 3.10.0-1160.49.1.el7.x86_64 x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:atomic-builtin
gcc_version:8.3.0
process_id:1
run_id:afaa34f1fda73418f2cd93f5ff43438f90bf6937
tcp_port:6379
uptime_in_seconds:771045
uptime_in_days:8
hz:10
configured_hz:10
lru_clock:15210632
executable:/redis-server
config_file:/opt/bitnami/redis/etc/redis.conf
io_threads_active:0

# Clients
connected_clients:95
client_recent_max_input_buffer:40960
client_recent_max_output_buffer:0
blocked_clients:25
tracking_clients:0
clients_in_timeout_table:25

# Memory
used_memory:12856608
used_memory_human:12.26M
used_memory_rss:30310400
used_memory_rss_human:28.91M
used_memory_peak:31560592
used_memory_peak_human:30.10M
used_memory_peak_perc:40.74%
used_memory_overhead:3609080
used_memory_startup:803816
used_memory_dataset:9247528
used_memory_dataset_perc:76.73%
allocator_allocated:13195272
allocator_active:15925248
allocator_resident:20357120
total_system_memory:33717776384
total_system_memory_human:31.40G
used_memory_lua:40960
used_memory_lua_human:40.00K
used_memory_scripts:2232
used_memory_scripts_human:2.18K
number_of_cached_scripts:5
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.21
allocator_frag_bytes:2729976
allocator_rss_ratio:1.28
allocator_rss_bytes:4431872
rss_overhead_ratio:1.49
rss_overhead_bytes:9953280
mem_fragmentation_ratio:2.34
mem_fragmentation_bytes:17332488
mem_not_counted_for_evict:0
mem_replication_backlog:0
mem_clients_slaves:0
mem_clients_normal:2050264
mem_aof_buffer:8
mem_allocator:jemalloc-5.1.0
active_defrag_running:0
lazyfree_pending_objects:0

# Persistence
loading:0
rdb_changes_since_last_save:41699953
rdb_bgsave_in_progress:0
rdb_last_save_time:1641829539
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:0
aof_enabled:1
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0
aof_current_size:92994702556
aof_base_size:86673312156
aof_pending_rewrite:0
aof_buffer_length:0
aof_rewrite_buffer_length:0
aof_pending_bio_fsync:0
aof_delayed_fsync:211

# Stats
total_connections_received:193651
total_commands_processed:44893160
instantaneous_ops_per_sec:8
total_net_input_bytes:286054105262
total_net_output_bytes:3295943667
instantaneous_input_kbps:231.44
instantaneous_output_kbps:0.04
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:78681
expired_stale_perc:0.10
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:39134
evicted_keys:0
keyspace_hits:6449104
keyspace_misses:1389430
pubsub_channels:14
pubsub_patterns:0
latest_fork_usec:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_reads_processed:56482100
total_writes_processed:20157894
io_threaded_reads_processed:0
io_threaded_writes_processed:0

# Replication
role:master
connected_slaves:0
master_replid:76939de367fe638c034dbf7518438c6575509b8f
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

# CPU
used_cpu_sys:1677.920262
used_cpu_user:7851.412872
used_cpu_sys_children:0.006463
used_cpu_user_children:0.004674

# Modules

# Cluster
cluster_enabled:0

# Keyspace
db0:keys=7994,expires=7119,avg_ttl=9973952903


127.0.0.1:6379>  config get auto-aof-rewrite-min-size
1) "auto-aof-rewrite-min-size"
2) "67108864"
127.0.0.1:6379>  config get auto-aof-rewrite-percentage
1) "auto-aof-rewrite-percentage"
2) "100"

It's the default bitnami charts used. With only one master. I have 2 installs of this prod/pre-prod, on the 2 install 92Go/61go for this file.

So each time I need to change redis of node, or restart redis it's take 30min atm for start.

I have to fix it and keep a file with good start time, go a appendonly file < 5/10go max.

Any idea to solve this on long-term ?

For information: Manual call of BGREWRITEAOF do nothing.

Thanks.

Answer 1

Try AOF with RDB-preamble. We got huge AOF file size decreases with it.

More details in this blog post.