PostgreSQL源码解析 on Last DBA

Parameters on the Control File and Primary-Standby Parameter Mismatch Issues

Mon, 25 Aug 2025 00:00:00 +0000

PARAMETER_CHANGE and Database Parameters on the Control File
#

Some PG parameters affect the standby’s operation. These parameters are not only in the configuration file but also written to the control file. Whenever parameters change, they are written to WAL and update the control file.

The standby redoes the PARAMETER_CHANGE WAL record and writes to the standby’s control file. PARAMETER_CHANGE WAL record:

rmgr: XLOG len (rec/tot): 54/ 54, tx: 0, lsn: 27F/800001C0, prev 27F/80000148, desc: PARAMETER_CHANGE max_connections=3000 max_worker_processes=20 max_wal_senders=10 max_prepared_xacts=0 max_locks_per_xact=1024 wal_level=logical wal_log_hints=off track_commit_timestamp=on

XLOG_PARAMETER_CHANGE records these 8 parameters, which can also be viewed directly from the control file:

$ pg_controldata |grep setting
wal_level setting: logical
wal_log_hints setting: on
max_connections setting: 1000
max_worker_processes setting: 20
max_wal_senders setting: 10
max_prepared_xacts setting: 0
max_locks_per_xact setting: 1024
track_commit_timestamp setting: on

These parameters are all from the primary, even if this control file belongs to the standby.

The startup process checks 6 parameters via the CheckRequiredParameterValues function. One parameter wal_level must be >= replica. The other 5 parameters — max_connections, max_worker_processes, max_wal_senders, max_prepared_transactions, max_locks_per_transaction — are checked for primary vs standby sizing. If the standby has a smaller value, recovery is paused. If you increase the primary’s parameters directly, the standby will crash. The PG log:

FATAL,22023,"hot standby is not possible because max_connections = 2000 is a lower setting than on the master server (its value was 3000)",,,,,"WAL redo at 27F/800001C0 for XLOG/PARAMETER_CHANGE: max_connections=3000 max_worker_processes=20 max_wal_senders=10 max_prepared_xacts=0 max_locks_per_xact=1024 wal_level=logical wal_log_hints=off track_commit_timestamp=on",,,,"","startup"

6 of the 8 parameters can seriously affect standby operation. The other 2 parameters — wal_log_hints, track_commit_timestamp — are not immediately checked by the startup process. All 8 parameters being synchronized to the control file serve their own purposes.

wal_log_hints Primary-Standby Mismatch
#

Changes to wal_log_hints are recorded in WAL logs. Although not checked by the startup process, pg_rewind does check it:

perform_rewind(...)
{
...
	/*
	 * Target cluster need to use checksums or hint bit wal-logging, this to
	 * prevent from data corruption that could occur because of hint bits.
	 */
	if (ControlFile_target.data_checksum_version != PG_DATA_CHECKSUM_VERSION &&
		!ControlFile_target.wal_log_hints)
	{
		pg_fatal("target server needs to use either data checksums or \"wal_log_hints = on\"");
	}

Since wal_log_hints is WAL-related, it doesn’t make sense for pg_rewind to check whether the standby’s wal_log_hints is enabled — it should check whether the primary’s wal_log_hints is enabled. Therefore, PG synchronizes the wal_log_hints parameter to the standby’s control file, which is very reasonable.

wal_log_hints primary-standby mismatch test:

select * from t1;
checkpoint;
update t1 set b='eee'; -- observation point 1
checkpoint; -- ignore this online checkpoint wal record
select * from t1; -- observation point 2

-- observation action
pg_waldump 000000020000027F0000000A|tail -10

-- observing option
select t_ctid,lp,
case lp_flags
when 0 then '0:LP_UNUSED'
when 1 then 'LP_NORMAL'
when 2 then 'LP_REDIRECT'
when 3 then 'LP_DEAD'
end as lp_flags,
t_xmin,t_xmax,t_field3 as t_cid, raw_flags, info.combined_flags,substring(t_data,0,40)
from heap_page_items(get_raw_page('t1',0)) item,
LATERAL heap_tuple_infomask_flags(t_infomask, t_infomask2) info
order by lp;

on, on:

-- Observation point 1:
rmgr: Heap len (rec/tot): 85/ 208, tx: 11140182, lsn: 27F/5000CC38, prev 27F/5000CBC0, desc: HOT_UPDATE off 3 xmax 11140182 flags 0x10 ; new off 4 xmax 0, blkref #0: rel 1663/7472552/7472597 blk 0 FPW
rmgr: Transaction len (rec/tot): 46/ 46, tx: 11140182, lsn: 27F/5000CD08, prev 27F/5000CC38, desc: COMMIT 2025-07-21 18:28:13.292397 CST
rmgr: Standby len (rec/tot): 50/ 50, tx: 0, lsn: 27F/5000CD38, prev 27F/5000CD08, desc: RUNNING_XACTS nextXid 11140183 latestCompletedXid 11140182 oldestRunningXid 11140183
-- Observation point 2:
rmgr: XLOG len (rec/tot): 51/ 171, tx: 0, lsn: 27F/58000110, prev 27F/580000D8, desc: FPI_FOR_HINT , blkref #0: rel 1663/7472552/7472597 blk 0 FPW
rmgr: Standby len (rec/tot): 50/ 50, tx: 0, lsn: 27F/580001C0, prev 27F/58000110, desc: RUNNING_XACTS nextXid 11140183 latestCompletedXid 11140182 oldestRunningXid 11140183

off, off:

-- Observation point 1:
rmgr: Heap len (rec/tot): 85/ 225, tx: 11140183, lsn: 27F/580003C8, prev 27F/58000390, desc: HOT_UPDATE off 4 xmax 11140183 flags 0x10 ; new off 5 xmax 0, blkref #0: rel 1663/7472552/7472597 blk 0 FPW
rmgr: Transaction len (rec/tot): 46/ 46, tx: 11140183, lsn: 27F/580004B0, prev 27F/580003C8, desc: COMMIT 2025-07-21 18:33:18.192146 CST
rmgr: Standby len (rec/tot): 50/ 50, tx: 0, lsn: 27F/580004E0, prev 27F/580004B0, desc: RUNNING_XACTS nextXid 11140184 latestCompletedXid 11140183 oldestRunningXid 11140184
rmgr: Standby len (rec/tot): 50/ 50, tx: 0, lsn: 27F/58000518, prev 27F/580004E0, desc: RUNNING_XACTS nextXid 11140184 latestCompletedXid 11140183 oldestRunningXid 11140184
-- Observation point 2:

on, off:

-- Observation point 1:
rmgr: Heap len (rec/tot): 85/ 274, tx: 11140186, lsn: 27F/58000C18, prev 27F/58000BA0, desc: HOT_UPDATE off 7 xmax 11140186 flags 0x10 ; new off 8 xmax 0, blkref #0: rel 1663/7472552/7472597 blk 0 FPW
rmgr: Transaction len (rec/tot): 46/ 46, tx: 11140186, lsn: 27F/58000D30, prev 27F/58000C18, desc: COMMIT 2025-07-21 18:40:17.638691 CST
rmgr: Standby len (rec/tot): 50/ 50, tx: 0, lsn: 27F/58000D60, prev 27F/58000D30, desc: RUNNING_XACTS nextXid 11140187 latestCompletedXid 11140186 oldestRunningXid 11140187
rmgr: Standby len (rec/tot): 50/ 50, tx: 0, lsn: 27F/58000D98, prev 27F/58000D60, desc: RUNNING_XACTS nextXid 11140187 latestCompletedXid 11140186 oldestRunningXid 11140187
-- Observation point 2:
rmgr: XLOG len (rec/tot): 51/ 236, tx: 0, lsn: 27F/58000E48, prev 27F/58000DD0, desc: FPI_FOR_HINT , blkref #0: rel 1663/7472552/7472597 blk 0 FPW
rmgr: Standby len (rec/tot): 50/ 50, tx: 0, lsn: 27F/58000F38, prev 27F/58000E48, desc: RUNNING_XACTS nextXid 11140187 latestCompletedXid 11140186 oldestRunningXid 11140187

off, on:

-- Observation point 1:
rmgr: Standby len (rec/tot): 50/ 50, tx: 0, lsn: 27F/58001108, prev 27F/58001090, desc: RUNNING_XACTS nextXid 11140187 latestCompletedXid 11140186 oldestRunningXid 11140187
rmgr: Heap len (rec/tot): 85/ 289, tx: 11140187, lsn: 27F/58001140, prev 27F/58001108, desc: HOT_UPDATE off 8 xmax 11140187 flags 0x10 ; new off 9 xmax 0, blkref #0: rel 1663/7472552/7472597 blk 0 FPW
rmgr: Transaction len (rec/tot): 46/ 46, tx: 11140187, lsn: 27F/58001268, prev 27F/58001140, desc: COMMIT 2025-07-21 18:44:08.550109 CST
rmgr: Standby len (rec/tot): 54/ 54, tx: 0, lsn: 27F/58001298, prev 27F/58001268, desc: RUNNING_XACTS nextXid 11140188 latestCompletedXid 11140186 oldestRunningXid 11140187; 1 xacts: 11140187
rmgr: Standby len (rec/tot): 50/ 50, tx: 0, lsn: 27F/580012D0, prev 27F/58001298, desc: RUNNING_XACTS nextXid 11140188 latestCompletedXid 11140187 oldestRunningXid 11140188
-- Observation point 2:

Test summary:

FPI_FOR_HINT is produced when hint bits are written back; SELECT queries can produce FPI_FOR_HINT.
Regardless of the standby setting (on or off), when the primary is on, FPI_FOR_HINT will be produced.

Additional Knowledge: What is XLOG_RUNNING_XACTS
#

XLOG_RUNNING_XACTS is one type of RM_STANDBY_ID:

/*
 * XLOG message types
 */
#define XLOG_STANDBY_LOCK			0x00
#define XLOG_RUNNING_XACTS			0x10
#define XLOG_INVALIDATIONS			0x20

XLOG_STANDBY_LOCK: Records acquisition and release of AccessExclusiveLock, used by standby nodes to recognize lock states.

XLOG_RUNNING_XACTS: Running-xacts snapshots used for building snapshots to ensure transaction consistency.

XLOG_INVALIDATIONS: INVALIDATIONS messages for synchronizing metadata information to local backends.

 * standbydefs.h
 *	 Frontend exposed definitions for hot standby mode.

RM_STANDBY_ID is an rmgr specifically defined for hot standby read-only standbys. For local instance recovery and logical decoding scenarios that need WAL, RM_STANDBY_ID is essentially meaningless to them.

Observing WAL records during transaction commit:

command	wal record
begin;
select * from txid_current(); –11140191
commit;	rmgr: Transaction len (rec/tot): 46/ 46, tx: 11140191, lsn: 27F/80000538, prev 27F/80000500, desc: COMMIT 2025-07-23 11:16:10.872724 CST
	rmgr: Standby len (rec/tot): 50/ 50, tx: 0, lsn: 27F/80000568, prev 27F/80000538, desc: RUNNING_XACTS nextXid 11140192 latestCompletedXid 11140191 oldestRunningXid 11140192

The transaction ID itself — commit or abort — is synchronized by rmgr: Transaction. Snapshots are synchronized via rmgr: Standby RUNNING_XACTS.

track_commit_timestamp Primary-Standby Mismatch
#

track_commit_timestamp: the startup process activates the standby’s commit_ts functionality upon receiving the corresponding WAL, primarily for viewing xid commit times on the standby:

/*
 * Activate or deactivate CommitTs' upon reception of a XLOG_PARAMETER_CHANGE
 * XLog record during recovery.
 */
void
CommitTsParameterChange(bool newvalue, bool oldvalue)
{
	/*
	 * If the commit_ts module is disabled in this server and we get word from
	 * the primary server that it is enabled there, activate it so that we can
	 * replay future WAL records involving it; also mark it as active on
	 * pg_control. If the old value was already set, we already did this, so
	 * don't do anything.
	 *
	 * If the module is disabled in the primary, disable it here too, unless
	 * the module is enabled locally.
	 *
	 * Note this only runs in the recovery process, so an unlocked read is
	 * fine.
	 */
	if (newvalue)
	{
		if (!commitTsShared->commitTsActive)
			ActivateCommitTs();
	}
	else if (commitTsShared->commitTsActive)
		DeactivateCommitTs();
}

track_commit_timestamp primary-standby mismatch test:

Initial state: primary=on, standby=on. Both can use committed_xact and similar functions.
primary=off (restart primary), standby=on (no change). Both cannot use committed_xact and similar functions.

After modifying and restarting the primary, standby replication remains normal, but committed_xact and similar functions are unusable:

$ select * from pg_last_committed_xact();
ERROR: 55000: could not get commit timestamp data
HINT: Make sure the configuration parameter "track_commit_timestamp" is set on the primary server.
LOCATION: error_commit_ts_disabled, commit_ts.c:385

$ show track_commit_timestamp
-> ;
 track_commit_timestamp
------------------------
 on
(1 row)

Time: 0.198 ms
$ \q
## pg_controldata |grep track_commit_timestamp
track_commit_timestamp setting: off

PG14+ Pause Recovery
#

PG14 improved the behavior when primary parameter changes cause standby crashes. When parameters don’t meet conditions, instead of the read-only standby directly crashing, it now only pauses replication. See RecoveryRequiresIntParameter.

Pause recovery on a hot standby server if the primary changes its parameters in a way that prevents replay on the standby (Peter Eisentraut)

Previously the standby would shut down immediately

Testing PG14 parameter changes causing standby replication interruption:

2025-07-23 19:46:31.337 CST,,,141823,,6880ca5f.229ff,14,,2025-07-23 19:41:19 CST,1/0,0,LOG,00000,"recovery has paused","If recovery is unpaused, the server will shut down.","You can then restart the server after making the necessary configuration changes.",,,"WAL redo at 281/78324BE8 for XLOG/PARAMETER_CHANGE: max_connections=2000 max_worker_processes=20 max_wal_senders=10 max_prepared_xacts=0 max_locks_per_xact=1024 wal_level=logical wal_log_hints=on track_commit_timestamp=on",,,,"","startup",,0

Since replication has already stopped, changing the primary’s parameters back won’t help — the standby can’t apply subsequent changes and update the control file. So you must modify the standby’s parameters and restart (the log hint is also quite clear).

Summary of the 8 Parameters
#

When any of the 8 parameters are modified on the primary and the primary is restarted, the local control file is updated. If parameters have changed, the updated parameters are written to WAL and synchronized to downstream. The downstream redoes this PARAMETER_CHANGE WAL record, updating its local control file. The standby then determines whether primary-standby replication or other functions are available based on certain conditions.

8 Parameters Written to Control File	Check	If not, standby (PG13-)	If not, standby (PG14+)
wal_level	!=minimal	Cannot sync, fundamental	Cannot sync, fundamental
max_connections	primary <= standby	hot standby shutdown	hot standby pause replication
max_worker_processes	primary <= standby	hot standby shutdown	hot standby pause replication
max_wal_senders	primary <= standby	hot standby shutdown	hot standby pause replication
max_prepared_transactions	primary <= standby	hot standby shutdown	hot standby pause replication
max_locks_per_transaction	primary <= standby	hot standby shutdown	hot standby pause replication
wal_log_hints	pg_rewind prerequisite (either data checksums or wal_log_hints = on)	Doesn’t affect standby sync	Doesn’t affect standby sync
track_commit_timestamp	Enable/disable standby commit_ts functionality	Doesn’t affect standby sync	Doesn’t affect standby sync

Special thanks to: Gao Changjun

A Brief Analysis of PostgreSQL TRUNCATE

Mon, 12 Aug 2024 00:00:00 +0000

Command Options
#

TRUNCATE [ TABLE ] [ ONLY ] name [ * ] [, ... ]
 [ RESTART IDENTITY | CONTINUE IDENTITY ] [ CASCADE | RESTRICT ]

1. ONLY: truncate only the specified table. When a table has inheritance children or child partitions, by default they are truncated together; ONLY can truncate just the inheritance parent table. Partitioned parent tables cannot specify ONLY.

-- Cannot truncate only a partitioned parent table
=> truncate only parttable;
ERROR: 42809: cannot truncate only a partitioned table
HINT: Do not specify the ONLY keyword, or use TRUNCATE ONLY on the partitions directly.
LOCATION: ExecuteTruncate, tablecmds.c:1655

-- truncate only the inheritance parent table, only the parent is cleaned
=> truncate table only parenttable;
TRUNCATE TABLE
=> select tableoid::regclass,count(*) from parenttable group by tableoid::regclass ;
 tableoid | count
------------+-------
 childtable | 1

-- Directly truncate the inheritance parent table, child tables are also cleaned
=> truncate table parenttable;
TRUNCATE TABLE
=> select tableoid::regclass,count(*) from parenttable group by tableoid::regclass ;
 tableoid | count
----------+-------
(0 rows)

2. RESTART IDENTITY CONTINUE IDENTITY: whether to reset sequences on columns. Default is CONTINUE.

-- bigserial creates a column sequence by default
=> create table tableserial (a bigserial not null,b name);
CREATE TABLE
=> \d+ tableserial;
 Table "public.tableserial"
 Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------+--------+-----------+----------+----------------------------------------+---------+--------------+-------------
 a | bigint | | not null | nextval('tableserial_a_seq'::regclass) | plain | |
 b | name | | | | plain | |

=> insert into tableserial(b) select md5(random()::text) from generate_series(1,1000);
INSERT 0 1000
-- seq current value is 1000
=> select currval('tableserial_a_seq'::regclass);
 currval
---------
 1000

-- Direct truncate does not reset sequences by default
=> truncate table tableserial;
TRUNCATE TABLE
=> select currval('tableserial_a_seq'::regclass) cur,nextval('tableserial_a_seq'::regclass);
 cur | nextval
------+---------
 1000 | 1001

-- Explicitly specify RESTART IDENTITY to reset sequences
=> truncate table tableserial RESTART IDENTITY;
TRUNCATE TABLE
-- Note: seq is reset on nextval
=> select currval('tableserial_a_seq'::regclass) cur,nextval('tableserial_a_seq'::regclass);
 cur | nextval
------+---------
 1001 | 1

3. CASCADE: truncate the table and all foreign key referencing tables.

-- Create primary table, foreign key table, and data
=> create table pri_tab(id bigint primary key,name varchar(10));
CREATE TABLE

=> insert into pri_tab values (1,'abc'),(2,'abc'),(3,'abc');
INSERT 0 3

=> create table frn_tab(id bigint,FOREIGN KEY (id) REFERENCES pri_tab(id));
CREATE TABLE

=> insert into frn_tab values (1),(2);
INSERT 0 2

=> select * from pri_tab;
 id | name
----+------
 1 | abc
 2 | abc
 3 | abc
(3 rows)

-- Foreign key table frn_tab depends on pri_tab's data
=> select * from frn_tab;
 id
----
 1
 2
(2 rows)

-- With foreign key references on the primary table, CASCADE is required on the foreign key table, otherwise truncate fails
=> truncate table pri_tab ;
ERROR: 0A000: cannot truncate a table referenced in a foreign key constraint
DETAIL: Table "frn_tab" references "pri_tab".
HINT: Truncate table "frn_tab" at the same time, or use TRUNCATE ... CASCADE.
LOCATION: heap_truncate_check_FKs, heap.c:3427

-- Clear foreign key constrained tables together
=> truncate table pri_tab cascade;
NOTICE: 00000: truncate cascades to table "frn_tab"
LOCATION: ExecuteTruncateGuts, tablecmds.c:1725
TRUNCATE TABLE
=> select * from pri_tab;
 id | name
----+------
(0 rows)

=> select * from frn_tab;
 id
----
(0 rows)

Since the foreign key table depends on the primary table’s data, you cannot directly truncate the primary table — you must add CASCADE, at which point the foreign key table is also cleared along with the primary table.

4. RESTRICT Whether to clear foreign key tables. Not very useful — it’s the default option, and behavior is the same whether specified or not. Use CASCADE to clear associated foreign key tables.

MVCC / Transaction
#

The PG official documentation has this passage:

TRUNCATE is not MVCC-safe. After truncation, the table will appear empty to concurrent transactions, if they are using a snapshot taken before the truncation occurred. TRUNCATE is transaction-safe with respect to the data in the tables: the truncation will be safely rolled back if the surrounding transaction does not commit.

transaction-safe means it can be placed inside a transaction block and can be rolled back. Rolling back truncate:

=> begin;
BEGIN
=> truncate t1;
TRUNCATE TABLE
=> rollback;
ROLLBACK
=> select count(*) from t1;
 count
-------
 100

not MVCC-safe means: if a session takes a snapshot before truncate, and a truncate occurs during the snapshot period, that snapshot can read the result after truncate. This does not conform to MVCC. However, this isn’t a big issue in session scenarios, because truncate takes an 8-level lock (AccessExclusiveLock). If the snapshot hasn’t ended, at minimum there’s a read shared lock on the table, so truncate won’t execute.

This will only be an issue for a transaction that did not access the table in question before the DDL command started — any transaction that has done so would hold at least an ACCESS SHARE table lock, which would block the DDL command until that transaction completes.

Feature Updates
#

There aren’t many truncate feature updates. Just note that PG14 added support for truncating foreign tables. The prerequisite for truncating foreign tables is that the FDW must support the TRUNCATE API.

Also it extends postgres_fdw so that it can issue TRUNCATE command to foreign servers, by adding new routine for that TRUNCATE API.

Functional Differences Between pg TRUNCATE and Other Databases
#

TRUNCATE being fast and an 8-level lock are already well-known traits. Compared to other databases, PG can also: choose whether to reset sequences (RESTART IDENTITY CONTINUE IDENTITY), rollback, and has simple authorization.

What TRUNCATE Does
#

create table lzl(a int);
create index lzl_idx on lzl(a);
create sequence lzl_seq start with 1;
alter table lzl alter column a set default nextval('lzl_seq');
--select pg_relation_filepath('lzl');

-- db path
=> select oid from pg_database where datname='lzldb';
 oid
--------
 418679

-- When first created, each rel's oid = relfilenode
=> select relname,oid,relfilenode,relkind from pg_class where relname like 'lzl%';
 relname | oid | relfilenode | relkind
---------+--------+-------------+---------
 lzl | 428363 | 428363 | r
 lzl_idx | 428366 | 428366 | i
 lzl_seq | 428367 | 428367 | S
(3 rows)

=> truncate table lzl;
TRUNCATE TABLE
=> select relname,oid,relfilenode,relkind from pg_class where relname like 'lzl%';
 relname | oid | relfilenode | relkind
---------+--------+-------------+---------
 lzl | 428363 | 428370 | r
 lzl_idx | 428366 | 428371 | i
 lzl_seq | 428367 | 428367 | S
-- After truncate, table and index were rebuilt, but sequence was not

M=> truncate table lzl RESTART IDENTITY;
TRUNCATE TABLE
=> select relname,oid,relfilenode,relkind from pg_class where relname like 'lzl%';
 relname | oid | relfilenode | relkind
---------+--------+-------------+---------
 lzl | 428363 | 428372 | r
 lzl_idx | 428366 | 428373 | i
 lzl_seq | 428367 | 428367 | S
-- Even with explicit RESTART, sequence was still not rebuilt

M=> alter sequence lzl_seq restart;
ALTER SEQUENCE
M=> select relname,oid,relfilenode,relkind from pg_class where relname like 'lzl%';
 relname | oid | relfilenode | relkind
---------+--------+-------------+---------
 lzl | 428363 | 428372 | r
 lzl_idx | 428366 | 428373 | i
 lzl_seq | 428367 | 428374 | S
-- Explicitly restarting the sequence DOES rebuild it

truncate ... RESTART IDENTITY did not rebuild our sequence, while alter sequence lzl_seq restart did rebuild the sequence. It seems the understanding of RESTART IDENTITY was wrong. Let’s look at the official documentation for RESTART IDENTITY:

Automatically restart sequences owned by columns of the truncated table(s).

The sequence must be owned by a column on the table — note: not owner to. Although \d shows sequences on the table, they may not belong to the table.

 \d+ lzl;
 Table "public.lzl"
 Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------+---------+-----------+----------+------------------------------+---------+--------------+-------------
 a | integer | | | nextval('lzl_seq'::regclass) | plain | |

Use owned by to modify the sequence’s owning table:

=> ALTER SEQUENCE lzl_seq OWNED BY lzl.a;
ALTER SEQUENCE

-- Check sequence owner information
SELECT s.relname AS seq, n.nspname AS sch, t.relname AS tab, a.attname AS col
FROM pg_class s
JOIN pg_depend d ON d.objid=s.oid AND d.classid='pg_class'::regclass AND d.refclassid='pg_class'::regclass
JOIN pg_class t ON t.oid=d.refobjid
JOIN pg_namespace n ON n.oid=t.relnamespace
JOIN pg_attribute a ON a.attrelid=t.oid AND a.attnum=d.refobjsubid
WHERE s.relkind='S' AND d.deptype='a';
 seq | sch | tab | col
-------------------+--------+-------------+-----
 tableserial_a_seq | public | tableserial | a
 lzl_seq | public | lzl | a

=> truncate table lzl RESTART IDENTITY;
TRUNCATE TABLE
M=> select relname,oid,relfilenode,relkind from pg_class where relname like 'lzl%';
 relname | oid | relfilenode | relkind
---------+--------+-------------+---------
 lzl | 428363 | 428375 | r
 lzl_idx | 428366 | 428376 | i
 lzl_seq | 428367 | 428377 | S

When a sequence is owned by a column on the table, explicitly specifying RESTART IDENTITY with truncate will restart that sequence, which also rebuilds the sequence. Sequences created via serial/bigserial are owned by the table and are dropped when the table is dropped; sequences not owned by a table are not dropped when the table is dropped.

Summary of truncate rebuild characteristics:

Direct truncate table rebuilds the table and indexes.
truncate table + RESTART IDENTITY rebuilds (i.e., restarts) sequences that belong to this table. If a sequence doesn’t belong to this table, even if the column’s default is associated with the sequence, the sequence won’t be rebuilt.

Source Code Analysis
#

TRUNCATE is also a utility command, and the entry function can be found quickly.

ExecuteTruncate in src/backend/commands/tablecmds.c is the entry function. The comments already explain that truncate must acquire an exclusive lock, check permissions and relation validity, and recursively check all tables that need to be truncated.

void
ExecuteTruncate(TruncateStmt *stmt)
{
...

/*
* Open, exclusive-lock, and check all the explicitly-specified relations
*/
foreach(cell, stmt->relations)
{
...
LOCKMODE lockmode = AccessExclusiveLock; // Level 8 lock
...

rel = table_open(myrelid, NoLock); // Open table

void
ExecuteTruncate(TruncateStmt *stmt)
{
...
	foreach(cell, stmt->relations)
	{
...
		LOCKMODE	lockmode = AccessExclusiveLock; // Level 8 lock
...
		/* open the relation, we already hold a lock on it */
		rel = table_open(myrelid, NoLock); // Open table
...
		truncate_check_activity(rel); // Even with the lock, verify it's not in use
...
		if (recurse) // Recursive execution
		{
...
			children = find_all_inheritors(myrelid, lockmode, NULL); // Find all inheritance children

			foreach(child, children)
			{
...
				 // Above only checked the parent table, recursion checks children
				truncate_check_rel(RelationGetRelid(rel), rel->rd_rel);
				truncate_check_activity(rel);

				rels = lappend(rels, rel); // Add to the list of rels to truncate
				relids = lappend_oid(relids, childrelid);
...
			}
		}
		// Recursion ends
		// truncate only on partitioned parent table? error directly
		else if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
			ereport(ERROR,
					(errcode(ERRCODE_WRONG_OBJECT_TYPE),
					 errmsg("cannot truncate only a partitioned table"),
					 errhint("Do not specify the ONLY keyword, or use TRUNCATE ONLY on the partitions directly.")));
	}

	// Main function
	ExecuteTruncateGuts(rels, relids, relids_logged,
						stmt->behavior, stmt->restart_seqs);

	/* And close the rels */
	foreach(cell, rels)
	{
		Relation	rel = (Relation) lfirst(cell);

		table_close(rel, NoLock);
	}
}

ExecuteTruncateGuts is called not only by the TRUNCATE command but also by the subscription side (publication/subscription can synchronize TRUNCATE).

void
ExecuteTruncateGuts(List *explicit_rels,
					List *relids,
					List *relids_logged,
					DropBehavior behavior, bool restart_seqs)
{
...
	rels = list_copy(explicit_rels);
	if (behavior == DROP_CASCADE) // If CASCADE option specified, extract all referencing relations
	{
		for (;;)
		{
...
			newrelids = heap_truncate_find_FKs(relids); // Find FKs
			if (newrelids == NIL)
				break;			/* nothing else to add */ // No rels, exit directly

			foreach(cell, newrelids)
			{
...
				rel = table_open(relid, AccessExclusiveLock); // All rels acquire AccessExclusiveLock
				ereport(NOTICE,
						(errmsg("truncate cascades to table \"%s\"",
								RelationGetRelationName(rel))));
				truncate_check_rel(relid, rel->rd_rel); // Check if it's a truncatable object — must be a data-storing table
				truncate_check_perms(relid, rel->rd_rel); // Check permissions
				truncate_check_activity(rel); // Check if in use
...
			}
		}
	}

...
	if (restart_seqs) // Handle restart seq
	{
		foreach(cell, rels)
		{
			Relation	rel = (Relation) lfirst(cell);
			List	 *seqlist = getOwnedSequences(RelationGetRelid(rel));
...
			// Only check sequence permissions
				if (!pg_class_ownercheck(seq_relid, GetUserId()))
					aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_SEQUENCE,
								 RelationGetRelationName(seq_rel));
...
		}
	}

...

	// Execute all BEFORE TRUNCATE triggers
	foreach(cell, rels)
	{
		ExecBSTruncateTriggers(estate, resultRelInfo);
		resultRelInfo++;
	}

// Begin the actual truncate
	foreach(cell, rels)
	{
...
		// If it's a partitioned parent table, do nothing
		if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
			continue;

		// Handle foreign tables
		if (rel->rd_rel->relkind == RELKIND_FOREIGN_TABLE)
		{
		...
		}

...
		 // If same transaction (may rollback), directly execute heap_truncate_one_rel without creating new relfilenode
		if (rel->rd_createSubid == mySubid ||
			rel->rd_newRelfilenodeSubid == mySubid)
		{
			/* Immediate, non-rollbackable truncation is OK */
			heap_truncate_one_rel(rel);
		}
		else
		{
...
			// Set NewRelfilenode
			RelationSetNewRelfilenode(rel, rel->rd_rel->relpersistence);

			heap_relid = RelationGetRelid(rel);

			 // Same for toast
			toast_relid = rel->rd_rel->reltoastrelid;
			if (OidIsValid(toast_relid))
			{
				Relation	toastrel = relation_open(toast_relid,
													 AccessExclusiveLock);

				RelationSetNewRelfilenode(toastrel,
										 toastrel->rd_rel->relpersistence);
				table_close(toastrel, NoLock);
			}

...
			 // Rebuild indexes
			reindex_relation(heap_relid, REINDEX_REL_PROCESS_TOAST,
							 &reindex_params);
		}

		pgstat_count_truncate(rel); // Update pgstat truncate count
	}

...
	// Reset sequences
	foreach(cell, seq_relids)
	{
		Oid			seq_relid = lfirst_oid(cell);

		ResetSequence(seq_relid);
	}

	// Write WAL
	if (list_length(relids_logged) > 0)
	{
	...
	}

	// Fire AFTER TRUNCATE triggers
	resultRelInfo = resultRelInfos;
	foreach(cell, rels)
	{
		ExecASTruncateTriggers(estate, resultRelInfo);
		resultRelInfo++;
	}
...
}

The ExecuteTruncateGuts function processes according to truncate options, with the following flow:

Find all referencing foreign key tables based on CASCADE option
Fire BEFORE TRUNCATE triggers
Execute truncate

If same transaction, don’t immediately create NewRelfilenode, directly call heap_truncate_one_rel for truncation
If not same transaction, call RelationSetNewRelfilenode to create new NewRelfilenode

reindex_relation function rebuilds indexes
Reset sequences based on RESTART IDENTITY
Write WAL log
Fire AFTER TRUNCATE triggers

Tracing further, there’s quite a bit of function nesting: RelationSetNewRelfilenode table_relation_set_new_filenode relation_set_new_filenode heapam_relation_set_new_filenode RelationCreateStorage Then to smgrcreate and smgr_create in src/backend/storage/smgr/smgr.c. The comment for smgr.c:

public interface routines to storage manager switch All file system operations in POSTGRES dispatch through these routines.

Any file system operation goes through smgr (storage manager); at this point it becomes file system operations.

Reference
#

https://www.postgresql.org/docs/15/sql-truncate.html https://www.postgresql.org/docs/current/mvcc-caveats.html https://pgpedia.info/t/truncate.html https://www.orafaq.com/wiki/SQL_FAQ https://learnsql.com/blog/difference-between-truncate-delete-and-drop-table-in-sql/

From Extremely Slow Unique Index Scan to Index Bloat

Mon, 12 Aug 2024 00:00:00 +0000

How Did a Primary Key Query Access Multiple Data Pages?
#

Continuing from the previous article: A Classic Case of Long Transactions, Table Bloat, and LIMIT Problems, there was one point not explained in detail:

Why does a query using the primary key generate so many shared hits? Why does index bloat cause access to multiple data pages? Can’t data outside the page be located through the corresponding index entry? This relates to index version management — indexes do carry some version information, but not much. Let’s first review PostgreSQL’s btree index structure.

（https://en.wikibooks.org/wiki/PostgreSQL/Index_Btree）

This PG btree wiki diagram doesn’t explain how dead tuples and dead index entries are accessed — it lacks version information. For now, you don’t need to understand every detail of this structure; just know that a btree structure like this exists.

To investigate the btree version access problem, let’s run a test:

create table tab1(a bigserial,b char(1000));
create index idx_tab1_a on tab1(a);
alter table tab1 set (autovacuum_enabled = off); --disable autovacuum
alter table tab1 alter column b set storage PLAIN; --disable toast

lzldb=> insert into tab1(b) values('zzzzzzzzz');
INSERT 0 1
--View tuple info on the data page
lzldb=> select t_ctid,lp,case lp_flags when 0 then 'LP_UNUSED' when 1 then 'LP_NORMAL' when 2 then 'LP_REDIRECT' when 3 then 'LP_DEAD' end as lp_flags,t_xmin,t_xmax,t_field3 as t_cid, raw_flags, info.combined_flags from heap_page_items(get_raw_page('tab1',0)) item,LATERAL heap_tuple_infomask_flags(t_infomask, t_infomask2) info order by lp;
 t_ctid | lp | lp_flags | t_xmin | t_xmax | t_cid | raw_flags | combined_flags 
--------+----+-----------+--------+--------+-------+--------------------------------------+----------------
 (0,1) | 1 | LP_NORMAL | 111875 | 0 | 0 | {HEAP_HASVARWIDTH,HEAP_XMAX_INVALID} | {}

--View index entry info on the index page (note: index page 0 is the meta page, has no data)
lzldb=> SELECT itemoffset, ctid, itemlen, nulls, vars, data, dead, htid, tids[0:2] AS some_tids FROM bt_page_items('idx_tab1_a',1);
 itemoffset | ctid | itemlen | nulls | vars | data | dead | htid | some_tids 
------------+-------+---------+-------+------+-------------------------+------+-------+-----------
 1 | (0,1) | 16 | f | f | 01 00 00 00 00 00 00 00 | f | (0,1) | 

Only one row inserted: data page 0 has only 1 tuple, index page 1 has only one entry pointing to ctid(0,1).

lzldb=> update tab1 set b='xxxxxxx' ;
UPDATE 1
lzldb=> select t_ctid,lp,case lp_flags when 0 then 'LP_UNUSED' when 1 then 'LP_NORMAL' when 2 then 'LP_REDIRECT' when 3 then 'LP_DEAD' end as lp_flags,t_xmin,t_xmax,t_field3 as t_cid, raw_flags, info.combined_flags from heap_page_items(get_raw_page('tab1',0)) item,LATERAL heap_tuple_infomask_flags(t_infomask, t_infomask2) info order by lp;
 t_ctid | lp | lp_flags | t_xmin | t_xmax | t_cid | raw_flags | combined_flags 
--------+----+-----------+--------+--------+-------+-------------------------------------------------------------------+----------------
 (0,2) | 1 | LP_NORMAL | 111875 | 111876 | 0 | {HEAP_HASVARWIDTH,HEAP_XMIN_COMMITTED,HEAP_HOT_UPDATED} | {}
 (0,2) | 2 | LP_NORMAL | 111876 | 0 | 0 | {HEAP_HASVARWIDTH,HEAP_XMAX_INVALID,HEAP_UPDATED,HEAP_ONLY_TUPLE} | {}
(2 rows)

lzldb=> SELECT itemoffset, ctid, itemlen, nulls, vars, data, dead, htid, tids[0:2] AS some_tids FROM bt_page_items('idx_tab1_a',1);
 itemoffset | ctid | itemlen | nulls | vars | data | dead | htid | some_tids 
------------+-------+---------+-------+------+-------------------------+------+-------+-----------
 1 | (0,1) | 16 | f | f | 01 00 00 00 00 00 00 00 | f | (0,1) | 

After updating one row: data page 0 has 2 tuples. Only ctid(0,2) is alive. The tuple at lp=1 is “dead” but lp_flags is still “NORMAL”! Index page 1 still has only one entry pointing to ctid(0,1), which is the “dead” tuple. This is the principle of HOT (Heap-Only Tuple): when updating within the same page, the index entry is not updated. The index follows the ctid chain from the dead tuple to find the truly alive data tuple.

Let’s update 10 times in a loop, producing 2 data pages and 1 index page:

DO $$
begin
	FOR i IN 1..10 LOOP
		update tab1 set b=md5(i::text);
	END LOOP; 
end $$;;

After updates:

--First data page
lzldb=> select t_ctid,lp,case lp_flags when 0 then 'LP_UNUSED' when 1 then 'LP_NORMAL' when 2 then 'LP_REDIRECT' when 3 then 'LP_DEAD' end as lp_flags,t_xmin,t_xmax,t_field3 as t_cid, raw_flags, info.combined_flags from heap_page_items(get_raw_page('tab1',0)) item,LATERAL heap_tuple_infomask_flags(t_infomask, t_infomask2) info order by lp;
 t_ctid | lp | lp_flags | t_xmin | t_xmax | t_cid | raw_flags | combined_flag
s 
--------+----+-------------+--------+--------+-------+--------------------------------------------------------------------------------------+--------------
--
 | 1 | LP_REDIRECT | | | | | 
 (0,3) | 2 | LP_NORMAL | 111876 | 111877 | 0 | {HEAP_HASVARWIDTH,HEAP_XMIN_COMMITTED,HEAP_UPDATED,HEAP_HOT_UPDATED,HEAP_ONLY_TUPLE} | {}
 (0,4) | 3 | LP_NORMAL | 111877 | 111877 | 0 | {HEAP_HASVARWIDTH,HEAP_COMBOCID,HEAP_UPDATED,HEAP_HOT_UPDATED,HEAP_ONLY_TUPLE} | {}
 (0,5) | 4 | LP_NORMAL | 111877 | 111877 | 1 | {HEAP_HASVARWIDTH,HEAP_COMBOCID,HEAP_UPDATED,HEAP_HOT_UPDATED,HEAP_ONLY_TUPLE} | {}
 (0,6) | 5 | LP_NORMAL | 111877 | 111877 | 2 | {HEAP_HASVARWIDTH,HEAP_COMBOCID,HEAP_UPDATED,HEAP_HOT_UPDATED,HEAP_ONLY_TUPLE} | {}
 (0,7) | 6 | LP_NORMAL | 111877 | 111877 | 3 | {HEAP_HASVARWIDTH,HEAP_COMBOCID,HEAP_UPDATED,HEAP_HOT_UPDATED,HEAP_ONLY_TUPLE} | {}
 (1,1) | 7 | LP_NORMAL | 111877 | 111877 | 4 | {HEAP_HASVARWIDTH,HEAP_COMBOCID,HEAP_UPDATED,HEAP_ONLY_TUPLE} | {}
(7 rows)
--Second data page
lzldb=> select t_ctid,lp,case lp_flags when 0 then 'LP_UNUSED' when 1 then 'LP_NORMAL' when 2 then 'LP_REDIRECT' when 3 then 'LP_DEAD' end as lp_flags,t_xmin,t_xmax,t_field3 as t_cid, raw_flags, info.combined_flags from heap_page_items(get_raw_page('tab1',1)) item,LATERAL heap_tuple_infomask_flags(t_infomask, t_infomask2) info order by lp;
 t_ctid | lp | lp_flags | t_xmin | t_xmax | t_cid | raw_flags | combined_flags 
--------+----+-----------+--------+--------+-------+--------------------------------------------------------------------------------+----------------
 (1,2) | 1 | LP_NORMAL | 111877 | 111877 | 5 | {HEAP_HASVARWIDTH,HEAP_COMBOCID,HEAP_UPDATED,HEAP_HOT_UPDATED} | {}
 (1,3) | 2 | LP_NORMAL | 111877 | 111877 | 6 | {HEAP_HASVARWIDTH,HEAP_COMBOCID,HEAP_UPDATED,HEAP_HOT_UPDATED,HEAP_ONLY_TUPLE} | {}
 (1,4) | 3 | LP_NORMAL | 111877 | 111877 | 7 | {HEAP_HASVARWIDTH,HEAP_COMBOCID,HEAP_UPDATED,HEAP_HOT_UPDATED,HEAP_ONLY_TUPLE} | {}
 (1,5) | 4 | LP_NORMAL | 111877 | 111877 | 8 | {HEAP_HASVARWIDTH,HEAP_COMBOCID,HEAP_UPDATED,HEAP_HOT_UPDATED,HEAP_ONLY_TUPLE} | {}
 (1,5) | 5 | LP_NORMAL | 111877 | 0 | 9 | {HEAP_HASVARWIDTH,HEAP_XMAX_INVALID,HEAP_UPDATED,HEAP_ONLY_TUPLE} | {}

On the first data page (page 0), the LP_REDIRECT status directly tells us the page definitely has HOT chains. At lp=1 there is no other information — not even ctid, data, or infomask. You cannot trace through this lp to find the final data. For the first index entry, it’s sufficient to access ctid(0,1); there is no desired data row in this page. But data page 2 has no LP_REDIRECT, and the index can find the live tuple (1,5) within the page by following the ctid chain from ctid(1,0).

Source code explanation of line pointer states:

/*
 *lp_flags has these possible states. An UNUSED line pointer is available
 *for immediate re-use, the other states are not.
 */
#define LP_UNUSED		0		/* unused (should always have lp_len=0) */
#define LP_NORMAL		1		/* used (should always have lp_len>0) */
#define LP_REDIRECT		2		/* HOT redirect (should have lp_len=0), actually not HOT but cross-page redirect indicator */
#define LP_DEAD			3		/* dead, may or may not have storage */

//Explanation of LP_REDIRECT
Redirecting line pointer

	A line pointer that points to another line pointer and has no
	associated tuple. It has the special lp_flags state LP_REDIRECT,
	and lp_off is the OffsetNumber of the line pointer it links to.
	This is used when a root tuple becomes dead but we cannot prune
	the line pointer because there are non-dead heap-only tuples
	further down the chain.

Looking back more carefully, the lp status of what we consider “dead” tuples is LP_NORMAL, not LP_DEAD. This is important because we’ll revisit this point later.

Continuing to examine the index page:

lzldb=> SELECT itemoffset, ctid, itemlen, nulls, vars, data, dead, htid, tids[0:2] AS some_tids FROM bt_page_items('idx_tab1_a',1);
 itemoffset | ctid | itemlen | nulls | vars | data | dead | htid | some_tids 
------------+-------+---------+-------+------+-------------------------+------+-------+-----------
 1 | (0,1) | 16 | f | f | 01 00 00 00 00 00 00 00 | f | (0,1) | 
 2 | (1,1) | 16 | f | f | 01 00 00 00 00 00 00 00 | f | (1,1) | 

Because an additional page was created, HOT no longer applies. The index is updated. The index page has only 2 entries, both alive (dead=f), each pointing to the first tuple of its respective page: (0,1) and (1,1). For cross-page updates, the index page is also updated, with each index entry pointing to its own page. Note: at this point the table has only 1 row of data, but the index has 2 entries, both alive. This is why a primary key scan accesses multiple data pages.

Let’s update more data to produce multiple index pages:

DO $$
begin
	FOR i IN 1..10000 LOOP
		update tab1 set b=md5(i::text);
	END LOOP; 
end $$;

--First index page
lzldb=> SELECT itemoffset, ctid, itemlen, nulls, vars, data, dead, htid, tids[0:2] AS some_tids FROM bt_page_items('idx_tab1_a',1);
 itemoffset | ctid | itemlen | nulls | vars | data | dead | htid | some_tids 
------------+-------------+---------+-------+------+-------------------------+------+----------+-------------------------
 1 | (1278,4097) | 24 | f | f | 01 00 00 00 00 00 00 00 | | (1277,1) | 
 2 | (16,8414) | 1352 | f | f | 01 00 00 00 00 00 00 00 | f | (0,1) | {"(0,1)","(1,1)"}
 3 | (16,8414) | 1352 | f | f | 01 00 00 00 00 00 00 00 | f | (222,1) | {"(222,1)","(223,1)"}
 4 | (16,8414) | 1352 | f | f | 01 00 00 00 00 00 00 00 | f | (444,1) | {"(444,1)","(445,1)"}
 5 | (16,8414) | 1352 | f | f | 01 00 00 00 00 00 00 00 | f | (666,1) | {"(666,1)","(667,1)"}
 6 | (16,8414) | 1352 | f | f | 01 00 00 00 00 00 00 00 | f | (888,1) | {"(888,1)","(889,1)"}

--Second index page
lzldb=> SELECT itemoffset, ctid, itemlen, nulls, vars, data, dead, htid, tids[0:2] AS some_tids FROM bt_page_items('idx_tab1_a',2);
 itemoffset | ctid | itemlen | nulls | vars | data | dead | htid | some_tids 
------------+----------+---------+-------+------+-------------------------+------+----------+-----------
 1 | (1278,1) | 16 | f | f | 01 00 00 00 00 00 00 00 | f | (1278,1) | 
 2 | (1279,1) | 16 | f | f | 01 00 00 00 00 00 00 00 | f | (1279,1) | 
 3 | (1280,1) | 16 | f | f | 01 00 00 00 00 00 00 00 | f | (1280,1) | 
 4 | (1281,1) | 16 | f | f | 01 00 00 00 00 00 00 00 | f | (1281,1) | 
...
 152 | (1429,1) | 16 | f | f | 01 00 00 00 00 00 00 00 | f | (1429,1) | 
 153 | (1430,1) | 16 | f | f | 01 00 00 00 00 00 00 00 | f | (1430,1) | 
(153 rows)

--Third index page
lzldb=> SELECT itemoffset, ctid, itemlen, nulls, vars, data, dead, htid, tids[0:2] AS some_tids FROM bt_page_items('idx_tab1_a',3);
 itemoffset | ctid | itemlen | nulls | vars | data | dead | htid | some_tids 
------------+----------+---------+-------+------+-------------------------+------+----------+-----------
 1 | (1,0) | 8 | f | f | | | | 
 2 | (2,4097) | 24 | f | f | 01 00 00 00 00 00 00 00 | | (1277,1) | 

There are 3 index pages total. Page 1 is the root node. Pages 2 and 3 are leaf nodes. The dead status of all their index entries is “f”.

Now let’s return to the SQL, using the primary key index:

lzldb=> explain (analyze,buffers) select * from tab1 where a=1;
 QUERY PLAN 
-----------------------------------------------------------------------------------------------------------------------
 Bitmap Heap Scan on tab1 (cost=4.39..56.41 rows=14 width=4012) (actual time=2.594..2.596 rows=1 loops=1)
 Recheck Cond: (a = 1)
 Heap Blocks: exact=1
 Buffers: shared hit=1437 dirtied=1026
 -> Bitmap Index Scan on idx_tab1_a (cost=0.00..4.39 rows=14 width=0) (actual time=0.152..0.153 rows=1431 loops=1)
 Index Cond: (a = 1)
 Buffers: shared hit=6
 Planning:
 Buffers: shared hit=5
 Planning Time: 0.087 ms
 Execution Time: 2.614 ms

When querying by primary key, shared hit is 1437, roughly matching the ~1430 table pages. Since indexes lack version information and the dead status of index entries hasn’t been updated, PostgreSQL follows all live index entries to find version information in the data pages. This is why a primary key index scan can be extremely slow.

kill index item
#

Since indexes don’t store visibility information (i.e., MVCC version info), the visibility of the tuple pointed to by an index determines the index visibility itself. This is also why index-only scans in PostgreSQL still access data pages. Of course, with the visibility map (VM), the VM records which data pages are all-visible and all-frozen, so index-only scans won’t access those pages — they’re already visible.

Even without VACUUM, the PostgreSQL kernel has a method for handling this kind of index bloat — kill index item. This feature is sometimes called Simple deletion or index deletion (terminology from src/backend/access/nbtree/README). Essentially, it marks index entries corresponding to tuples that are already LP_DEAD as dead, without changing the existing index structure.

Source code function _bt_killitems:

* _bt_killitems - set LP_DEAD state for items an indexscan caller has
* told us were killed

This clearly states that index scans trigger kill item operations (meaning SELECT can also trigger this operation to update the index). This is easy to test. Since our previous data has already been index-scanned, let’s rebuild data for testing.

create table tab2(a bigserial,b char(100));
create index idx_tab2_a on tab2(a);
create index idx_tab2_b on tab2(b);
alter table tab2 set (autovacuum_enabled = off); --disable autovacuum
alter table tab2 alter column b set storage PLAIN; --disable toast
--Insert 1 row and update repeatedly
insert into tab2(b) values('00000');

DO $$
begin
	FOR i IN 1..10000 LOOP
		update tab2 set b=i::text;
	END LOOP; 
end $$;

--Table pages
lzldb=> select t_ctid,lp,case lp_flags when 0 then 'LP_UNUSED' when 1 then 'LP_NORMAL' when 2 then 'LP_REDIRECT' when 3 then 'LP_DEAD' end as lp_flags,t_xmin,t_xmax,t_field3 as t_cid, raw_flags, info.combined_flags from heap_page_items(get_raw_page('tab2',2)) item,LATERAL heap_tuple_infomask_flags(t_infomask, t_infomask2) info order by lp;
 t_ctid | lp | lp_flags | t_xmin | t_xmax | t_cid | raw_flags | combined_flags 
--------+----+-----------+--------+--------+-------+-----------------------------------------------+----------------
 (2,2) | 1 | LP_NORMAL | 509 | 509 | 115 | {HEAP_HASVARWIDTH,HEAP_COMBOCID,HEAP_UPDATED} | {}
 (2,3) | 2 | LP_NORMAL | 509 | 509 | 116 | {HEAP_HASVARWIDTH,HEAP_COMBOCID,HEAP_UPDATED} | {}
 (2,4) | 3 | LP_NORMAL | 509 | 509 | 117 | {HEAP_HASVARWIDTH,HEAP_COMBOCID,HEAP_UPDATED} | {}
 (2,5) | 4 | LP_NORMAL | 509 | 509 | 118 | {HEAP_HASVARWIDTH,HEAP_COMBOCID,HEAP_UPDATED} | {}
 (2,6) | 5 | LP_NORMAL | 509 | 509 | 119 | {HEAP_HASVARWIDTH,HEAP_COMBOCID,HEAP_UPDATED} | {}
 (2,7) | 6 | LP_NORMAL | 509 | 509 | 120 | {HEAP_HASVARWIDTH,HEAP_COMBOCID,HEAP_UPDATED} | {}
 (2,8) | 7 | LP_NORMAL | 509 | 509 | 121 | {HEAP_HASVARWIDTH,HEAP_COMBOCID,HEAP_UPDATED} | {}
...

--Index a pages
lzldb=> SELECT itemoffset, ctid, itemlen, nulls, vars, dead, htid, tids[0:2] AS some_tids FROM bt_page_items('idx_tab2_a',4);
 itemoffset | ctid | itemlen | nulls | vars | dead | htid | some_tids 
------------+-----------+---------+-------+------+------+---------+-----------------------
 1 | (66,4097) | 24 | f | f | | (66,6) | 
 2 | (16,8414) | 1352 | f | f | f | (44,5) | {"(44,5)","(44,6)"}
 3 | (16,8414) | 1352 | f | f | f | (47,53) | {"(47,53)","(47,54)"}
 4 | (16,8414) | 1352 | f | f | f | (51,43) | {"(51,43)","(51,44)"}
 5 | (16,8414) | 1352 | f | f | f | (55,33) | {"(55,33)","(55,34)"}
 6 | (16,8414) | 1352 | f | f | f | (59,23) | {"(59,23)","(59,24)"}
 7 | (16,8360) | 1024 | f | f | f | (63,13) | {"(63,13)","(63,14)"}
--Index b pages
lzldb=> SELECT itemoffset, ctid, itemlen, nulls, vars, dead, htid, tids[0:2] AS some_tids FROM bt_page_items('idx_tab2_b',4);
 itemoffset | ctid | itemlen | nulls | vars | dead | htid | some_tids 
------------+---------+---------+-------+------+------+---------+-----------
 1 | (57,1) | 112 | f | t | | | 
 2 | (0,34) | 112 | f | t | f | (0,34) | 
 3 | (5,41) | 112 | f | t | f | (5,41) | 
 4 | (56,53) | 112 | f | t | f | (56,53) | 
 5 | (56,54) | 112 | f | t | f | (56,54) | 
 6 | (56,55) | 112 | f | t | f | (56,55) | 
 7 | (56,56) | 112 | f | t | f | (56,56) | 
 8 | (56,57) | 112 | f | t | f | (56,57) | 

Now query the table with a sequential scan, then examine the data tuple and index entry states:

lzldb=> explain (analyze,buffers) select * from tab2;
 QUERY PLAN 
-----------------------------------------------------------------------------------------------------
 Seq Scan on tab2 (cost=0.00..204.14 rows=3114 width=412) (actual time=1.077..1.079 rows=1 loops=1)
 Buffers: shared hit=173 dirtied=173
 Planning Time: 0.042 ms
 Execution Time: 1.090 ms
lzldb=> select t_ctid,lp,case lp_flags when 0 then 'LP_UNUSED' when 1 then 'LP_NORMAL' when 2 then 'LP_REDIRECT' when 3 then 'LP_DEAD' end as lp_flags,t_xmin,t_xmax,t_field3 as t_cid, raw_flags, info.combined_flags from heap_page_items(get_raw_page('tab2',4)) item,LATERAL heap_tuple_infomask_flags(t_infomask, t_infomask2) info order by lp;
 t_ctid | lp | lp_flags | t_xmin | t_xmax | t_cid | raw_flags | combined_flags 
--------+----+----------+--------+--------+-------+-----------+----------------
 | 1 | LP_DEAD | | | | | 
 | 2 | LP_DEAD | | | | | 
 | 3 | LP_DEAD | | | | | 
 | 4 | LP_DEAD | | | | | 
 | 5 | LP_DEAD | | | | | 
 | 6 | LP_DEAD | | | | | 
 | 7 | LP_DEAD | | | | | 

lzldb=> SELECT itemoffset, ctid, itemlen, nulls, vars, dead, htid, tids[0:2] AS some_tids FROM bt_page_items('idx_tab2_a',4);
 itemoffset | ctid | itemlen | nulls | vars | dead | htid | some_tids 
------------+-----------+---------+-------+------+------+---------+-----------------------
 1 | (66,4097) | 24 | f | f | | (66,6) | 
 2 | (16,8414) | 1352 | f | f | f | (44,5) | {"(44,5)","(44,6)"}
 3 | (16,8414) | 1352 | f | f | f | (47,53) | {"(47,53)","(47,54)"}
 4 | (16,8414) | 1352 | f | f | f | (51,43) | {"(51,43)","(51,44)"}
 5 | (16,8414) | 1352 | f | f | f | (55,33) | {"(55,33)","(55,34)"}
 6 | (16,8414) | 1352 | f | f | f | (59,23) | {"(59,23)","(59,24)"}
 7 | (16,8360) | 1024 | f | f | f | (63,13) | {"(63,13)","(63,14)"}
(7 rows)

lzldb=> SELECT itemoffset, ctid, itemlen, nulls, vars, dead, htid, tids[0:2] AS some_tids FROM bt_page_items('idx_tab2_b',4);
 itemoffset | ctid | itemlen | nulls | vars | dead | htid | some_tids 
------------+---------+---------+-------+------+------+---------+-----------
 1 | (57,1) | 112 | f | t | | | 
 2 | (0,34) | 112 | f | t | f | (0,34) | 
 3 | (5,41) | 112 | f | t | f | (5,41) | 
 4 | (56,53) | 112 | f | t | f | (56,53) | 
 5 | (56,54) | 112 | f | t | f | (56,54) | 
 6 | (56,55) | 112 | f | t | f | (56,55) | 
 7 | (56,56) | 112 | f | t | f | (56,56) | 

Data tuples: all pages except the last were marked LP_DEAD. Index entries: nothing changed.

Now query again using index a:

lzldb=> explain (analyze,buffers) select * from tab2 where a=1;
 QUERY PLAN 
---------------------------------------------------------------------------------------------------------------------
 Index Scan using idx_tab2_a on tab2 (cost=0.28..68.56 rows=16 width=412) (actual time=1.282..1.510 rows=1 loops=1)
 Index Cond: (a = 1)
 Buffers: shared hit=190 dirtied=8
 Planning Time: 0.058 ms
 Execution Time: 1.525 ms
(5 rows)

lzldb=> select t_ctid,lp,case lp_flags when 0 then 'LP_UNUSED' when 1 then 'LP_NORMAL' when 2 then 'LP_REDIRECT' when 3 then 'LP_DEAD' end as lp_flags,t_xmin,t_xmax,t_field3 as t_cid, raw_flags, info.combined_flags from heap_page_items(get_raw_page('tab2',0)) item,LATERAL heap_tuple_infomask_flags(t_infomask, t_infomask2) info order by lp;
 t_ctid | lp | lp_flags | t_xmin | t_xmax | t_cid | raw_flags | combined_flags 
--------+----+----------+--------+--------+-------+-----------+----------------
 | 1 | LP_DEAD | | | | | 
 | 2 | LP_DEAD | | | | | 
 | 3 | LP_DEAD | | | | | 
 | 4 | LP_DEAD | | | | | 
 | 5 | LP_DEAD | | | | | 
 | 6 | LP_DEAD | | | | | 
lzldb=> SELECT itemoffset, ctid, itemlen, nulls, vars, dead, htid, tids[0:2] AS some_tids FROM bt_page_items('idx_tab2_a',4);
 itemoffset | ctid | itemlen | nulls | vars | dead | htid | some_tids 
------------+-----------+---------+-------+------+------+---------+-----------------------
 1 | (66,4097) | 24 | f | f | | (66,6) | 
 2 | (16,8414) | 1352 | f | f | t | (44,5) | {"(44,5)","(44,6)"}
 3 | (16,8414) | 1352 | f | f | t | (47,53) | {"(47,53)","(47,54)"}
 4 | (16,8414) | 1352 | f | f | t | (51,43) | {"(51,43)","(51,44)"}
 5 | (16,8414) | 1352 | f | f | t | (55,33) | {"(55,33)","(55,34)"}
 6 | (16,8414) | 1352 | f | f | t | (59,23) | {"(59,23)","(59,24)"}
 7 | (16,8360) | 1024 | f | f | t | (63,13) | {"(63,13)","(63,14)"}
(7 rows)

lzldb=> SELECT itemoffset, ctid, itemlen, nulls, vars, dead, htid, tids[0:2] AS some_tids FROM bt_page_items('idx_tab2_b',4);
 itemoffset | ctid | itemlen | nulls | vars | dead | htid | some_tids 
------------+---------+---------+-------+------+------+---------+-----------
 1 | (57,1) | 112 | f | t | | | 
 2 | (0,34) | 112 | f | t | f | (0,34) | 
 3 | (5,41) | 112 | f | t | f | (5,41) | 
 4 | (56,53) | 112 | f | t | f | (56,53) | 
 5 | (56,54) | 112 | f | t | f | (56,54) | 
 6 | (56,55) | 112 | f | t | f | (56,55) | 
 7 | (56,56) | 112 | f | t | f | (56,56) | 

The dead tuples in index a have all been marked dead=t, while dead tuples in index b remain dead=f because we haven’t scanned index b.

Now query through index a again:

lzldb=> explain (analyze,buffers) select * from tab2 where a=1;
 QUERY PLAN 
---------------------------------------------------------------------------------------------------------------------
 Index Scan using idx_tab2_a on tab2 (cost=0.28..68.56 rows=16 width=412) (actual time=0.020..0.021 rows=1 loops=1)
 Index Cond: (a = 1)
 Buffers: shared hit=10
 Planning Time: 0.059 ms
 Execution Time: 0.033 ms

Because the index entries for dead tuples in index a have been marked dead=t, there’s no need to check version information on data pages to determine whether tuples are “alive.”

Why is shared hit=10 here, still somewhat high? Because kill index item only marks dead index entries without changing the index structure, so the number of index pages hasn’t decreased. These 10 shared hits correspond to 10 index pages (including the meta page).

lzldb=> analyze tab2;
ANALYZE
lzldb=> select relname,relpages,reltuples from pg_class where relname='idx_tab2_a';
 relname | relpages | reltuples 
------------+----------+-----------
 idx_tab2_a | 10 | 1

Bottom-Up deletion
#

In PG14, the trigger condition for index deletion was enhanced. As mentioned earlier, index deletion is triggered by scanning the index. In PG14, index deletion can also be triggered when an index page split is imminent, to find free index space and reduce the probability of page splits.

This feature reduces index splits and thus also reduces index bloat, mitigating the problems caused by index bloat.

For specific testing, see: INDEX BLOAT REDUCED IN POSTGRESQL V14

index deduplication
#

PG13 introduced the index deduplication feature, which brings the GIN index posting list concept into btree indexes to reduce the space occupied by duplicate btree index entries and mitigate index split issues.

Previously, btree index entries pointed to only one ctid (as we saw in the tests above). With deduplicate index items, one index entry can have a posting list, and one posting list can hold multiple ctids.

The representation of posting lists is almost identical to the posting lists used by GIN

Like GIN posting tree(list) (the btree posting list may not exactly follow this structure — needs further study):

（https://postgrespro.com/blog/pgsql/4261647）

Testing index deduplication:

create table tab3(same char(100),diff char(100));
create index idx_tab3_same on tab3(same);
create index idx_tab3_diff on tab3(diff);
insert into tab3 select 10000::text,i::text from generate_series(10000, 99999) as i;

lzldb=> SELECT itemoffset, ctid, itemlen, nulls, vars, dead, htid, tids[0:2] AS some_tids FROM bt_page_items('idx_tab3_same',4);
 itemoffset | ctid | itemlen | nulls | vars | dead | htid | some_tids 
------------+------------+---------+-------+------+------+----------+-----------------------
 1 | (104,4097) | 120 | f | t | | (104,10) | 
 2 | (112,8398) | 1352 | f | t | f | (69,19) | {"(69,19)","(69,20)"}
 3 | (112,8398) | 1352 | f | t | f | (75,21) | {"(75,21)","(75,22)"}
 4 | (112,8398) | 1352 | f | t | f | (81,23) | {"(81,23)","(81,24)"}
 5 | (112,8398) | 1352 | f | t | f | (87,25) | {"(87,25)","(87,26)"}
 6 | (112,8398) | 1352 | f | t | f | (93,27) | {"(93,27)","(93,28)"}
 7 | (112,8344) | 1024 | f | t | f | (99,29) | {"(99,29)","(99,30)"}
(7 rows)

lzldb=> SELECT itemoffset, ctid, itemlen, nulls, vars, dead, htid, tids[0:2] AS some_tids FROM bt_page_items('idx_tab3_diff',4);
 itemoffset | ctid | itemlen | nulls | vars | dead | htid | some_tids 
------------+--------+---------+-------+------+------+--------+-----------
 1 | (5,1) | 112 | f | t | | | 
 2 | (3,23) | 112 | f | t | f | (3,23) | 
 3 | (3,24) | 112 | f | t | f | (3,24) | 
... 
 62 | (5,15) | 112 | f | t | f | (5,15) | 
 63 | (5,16) | 112 | f | t | f | (5,16) | 
(63 rows) 

The tids column in the bt_page_items function is essentially the posting list. The same field was inserted with identical data and produced deduplication in the index; the diff field had no duplicate data and produced no deduplication.

The space difference is enormous:

lzldb=> select relname,relpages,reltuples from pg_class where relname like 'idx_tab3%';
 relname | relpages | reltuples 
---------------+----------+-----------
 idx_tab3_diff | 1484 | 90000
 idx_tab3_same | 81 | 90000

Can unique indexes produce deduplication?
#

Unique indexes have no duplicate data, so it seems like they wouldn’t. In practice, they can. Because even with unique indexes, when HOT can’t satisfy an update, multiple index entries are created. We can see this from the first test case in this article. Repeatedly updating a single row with UPDATE also produces deduplication, which occurs before delete index item.

Additionally, when delete index item removes a posting list index entry, it must ensure that all ctids under the posting list correspond to DEAD tuples.

Disabling deduplication
#

Index deduplication was introduced in PG13. The feature is enabled by default and can be disabled at the index level. Modifying deduplicate_items on an index won’t directly change the existing index structure; it only affects newly inserted data.

alter index idx_tab3_same set (deduplicate_items=off);
create index idx_tab3_same1 on tab3(same) with (deduplicate_items=off);

What does VACUUM do?
#

VACUUM does many things. Here we’ll only focus on table/index bloat and space reclamation, skipping wraparound and other topics.

Let’s test with tab2, where we repeatedly updated a single row. Simple deletion has already been triggered, and table/index entries are almost all DEAD.

Run VACUUM directly:

lzldb=# vacuum verbose tab2;
INFO: vacuuming "public.tab2"
INFO: scanned index "idx_tab2_a" to remove 10000 row versions
DETAIL: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
INFO: scanned index "idx_tab2_b" to remove 10000 row versions
DETAIL: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
INFO: "tab2": removed 10000 row versions in 173 pages
DETAIL: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
INFO: index "idx_tab2_a" now contains 1 row versions in 10 pages
DETAIL: 10000 index row versions were removed.
7 index pages have been deleted, 0 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: index "idx_tab2_b" now contains 1 row versions in 276 pages
DETAIL: 10000 index row versions were removed.
269 index pages have been deleted, 0 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: "tab2": found 24 removable, 1 nonremovable row versions in 173 out of 173 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 526
There were 0 unused item identifiers.
Skipped 0 pages due to buffer pins, 0 frozen pages.
0 pages are entirely empty.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
VACUUM

idx_tab2_a removed 10000 row versions in 10 pages, 7 index pages were deleted. Table tab2 removed 10000 row versions in 173 pages.

--First page of the table
lzldb=> select t_ctid,lp,case lp_flags when 0 then 'LP_UNUSED' when 1 then 'LP_NORMAL' when 2 then 'LP_REDIRECT' when 3 then 'LP_DEAD' end as lp_flags,t_xmin,t_xmax,t_field3 as t_cid, raw_flags, info.combined_flags from heap_page_items(get_raw_page('tab2',0)) item,LATERAL heap_tuple_infomask_flags(t_infomask, t_infomask2) info order by lp;
 t_ctid | lp | lp_flags | t_xmin | t_xmax | t_cid | raw_flags | combined_flags 
--------+----+-----------+--------+--------+-------+-----------+----------------
 | 1 | LP_UNUSED | | | | | 
 | 2 | LP_UNUSED | | | | | 
...
 | 45 | LP_UNUSED | | | | | 

--Last page of the table
lzldb=> select t_ctid,lp,case lp_flags when 0 then 'LP_UNUSED' when 1 then 'LP_NORMAL' when 2 then 'LP_REDIRECT' when 3 then 'LP_DEAD' end as lp_flags,t_xmin,t_xmax,t_field3 as t_cid, raw_flags, info.combined_flags from heap_page_items(get_raw_page('tab2',172)) item,LATERAL heap_tuple_infomask_flags(t_infomask, t_infomask2) info order by lp;
 t_ctid | lp | lp_flags | t_xmin | t_xmax | t_cid | raw_flags | combined_flags 
----------+----+-----------+--------+--------+-------+-----------------------------------------------------------------------+----------------
 | 1 | LP_UNUSED | | | | | 
 | 2 | LP_UNUSED | | | | | 
...
 | 23 | LP_UNUSED | | | | | 
 | 24 | LP_UNUSED | | | | | 
 (172,25) | 25 | LP_NORMAL | 509 | 0 | 9999 | {HEAP_HASVARWIDTH,HEAP_XMIN_COMMITTED,HEAP_XMAX_INVALID,HEAP_UPDATED} | {}

--First index page
lzldb=> SELECT itemoffset, ctid, itemlen, nulls, vars, dead, htid, tids[0:2] AS some_tids FROM bt_page_items('idx_tab2_a',1);
NOTICE: page is deleted
 itemoffset | ctid | itemlen | nulls | vars | dead | htid | some_tids 
------------+----------------+---------+-------+------+------+------+-----------
 1 | (4294967295,0) | 8 | f | f | | | 
(1 row)

--Last index page
lzldb=> SELECT itemoffset, ctid, itemlen, nulls, vars, dead, htid, tids[0:2] AS some_tids FROM bt_page_items('idx_tab2_a',9);
 itemoffset | ctid | itemlen | nulls | vars | dead | htid | some_tids 
------------+----------+---------+-------+------+------+----------+-----------
 1 | (172,25) | 16 | f | f | f | (172,25) | 

All line pointers for dead table tuples were marked UNUSED, data was cleaned, and only one live tuple remains in NORMAL state. The table still has the same number of pages.

All dead index entries (dead=t) were cleaned. Live index entries were shifted within index pages (the last page’s index entry originally had itemoffset != 1). All emptied index pages were marked as deleted. These deleted index pages still exist, in a half-dead state.

From the nbtree README on “Deleting entire pages during VACUUM” (the original is quite long; I’ve excerpted the key parts):

We consider deleting an entire page from the btree only when it’s become completely empty of items. Page deletion always begins from an empty leaf page. An internal page can only be deleted as part of deleting an entire subtree.

An entire page is only considered for deletion when the index page is completely empty. Deletion always starts from leaf nodes; non-leaf nodes are only deleted when deleting an entire subtree.

Deleting a leaf page is a two-stage process.
In the first stage, the page is unlinked from its parent, and marked as half-dead. In the second-stage, the half-dead leaf page is unlinked from its siblings. We first lock the left sibling (if any) of the target, the target page itself, and its right sibling (there must be one) in that order. Then we update the side-links in the siblings, and mark the target page deleted.

Deleting a leaf page has two stages:

Unlink from the parent — the leaf page is now in half-dead state
Unlink from left and right siblings — the leaf page is now in deleted state

A deleted page cannot be recycled immediately, since there may be other processes waiting to reference it (ie, search processes that just left the parent, or scans moving right or left from one of the siblings). These processes must be able to observe a deleted page for some time after the deletion operation, in order to be able to at least recover from it (they recover by moving right, as with concurrent page splits). Searchers never have to worry about concurrent page recycling.

Because other processes may still be using the deleted page, VACUUM cannot immediately recycle these index pages.

This description matches what we observed.

Although after VACUUM, the index still has the same number of pages:

 relname | relpages | reltuples 
------------+----------+-----------
 idx_tab2_a | 10 | 1
 tab2 | 173 | 1

The index scan no longer needs to access deleted pages:

lzldb=> explain (analyze,buffers) select * from tab2 where a=1; QUERY PLAN 
-------------------------------------------------------------------------------------------------------------------
 Index Scan using idx_tab2_a on tab2 (cost=0.12..8.14 rows=1 width=109) (actual time=0.011..0.012 rows=1 loops=1)
 Index Cond: (a = 1)
 Buffers: shared hit=2
 Planning Time: 0.056 ms
 Execution Time: 0.025 ms

Before VACUUM, shared hit=10. After VACUUM, the number of index pages hasn’t changed — still 10, with 8 pages deleted but not directly recycled, so shared hit=2. Why 2 is easy to understand: “meta page” + “the one surviving leaf page.”

Placing deleted pages in the FSM
#

Recycling a page is decoupled from page deletion. A deleted page can only be put in the FSM to be recycled once there is no possible scan or search that has a reference to it; until then, it must stay in place with its sibling links undisturbed, as a tombstone that allows concurrent searches to detect and then recover from concurrent deletions (which are rather like concurrent page splits to searchers)

What is “Placing deleted pages in the FSM”? After an index page is deleted, it isn’t directly recycled. During index splits or new page allocation, it’s hard to find deleted pages for reuse. Placing deleted pages in the FSM puts these recyclable pages into the index’s corresponding FSM file, making it easy to find available free pages.

As mentioned earlier, during the first VACUUM, those deleted pages are unlinked but still occupy space. Before PG14:

We implement the technique by waiting until all active snapshots and registered snapshots as of the page deletion are gone

One condition for deletion: all active snapshots and snapshots related to the deleted pages must have ended. So long transactions definitely affect placing.

Placing an already-deleted page in the FSM to be recycled when needed doesn’t actually change the state of the page. The page will be changed whenever it is subsequently taken from the FSM for reuse. The deleted page’s contents will be overwritten by the split operation (it will become the new right sibling page).

Additionally, putting an already-deleted page into the FSM file doesn’t change the page’s state — this is just to quickly locate available free pages.

Prior to PostgreSQL 14, VACUUM would only place old deleted pages that it encounters during its linear scan (pages deleted by a previous VACUUM operation) in the FSM. Newly deleted pages were never placed in the FSM, because that was assumed to always be unsafe. PostgreSQL 14 added the ability for VACUUM to consider if it’s possible to recycle newly deleted pages at the end of the full index scan where the page deletion took place

Before PG14, deleted pages produced by the first VACUUM were not placed in the FSM. Only “old” deleted pages would be placed in the FSM file. Starting from PG14, the first VACUUM also considers placing deleted pages in the FSM.

Test (my version is PG13):

The tab2 test above just ran one VACUUM. Although deleted pages were produced, the index has no corresponding FSM file:

lzldb=> select * from pg_relation_filepath('idx_tab2_a');
 pg_relation_filepath 
----------------------
 base/16384/16437
 
[postgres@lzlhost data]$ ll base/16384/16437*
-rw------- 1 postgres postgres 81920 Apr 5 11:04 base/16384/16437

Now run VACUUM again:

lzldb=> vacuum tab2;

[postgres@lzlhost data]$ ll base/16384/16437*
-rw------- 1 postgres postgres 81920 Apr 5 11:04 base/16384/16437
-rw------- 1 postgres postgres 24576 Apr 5 15:52 base/16384/16437_fsm

The index immediately generated an FSM file.

Flowchart: Index Bloat and Cleanup
#

Please note:

The diagram below does not include table FSM/VM information
The diagram below does not include deduplication information
Version is PG13

fillfactor
#

Above we covered various kernel-supported methods for reducing index bloat. Beyond these approaches that require little active participation, you can also adjust table and index fillfactor to control bloat.

Fillfactor is essentially the waterline for tables or indexes. When INSERTING data, once the page reaches the fillfactor line, insertion moves to the next page. Fillfactor is designed to leave room for UPDATE operations, preventing UPDATE from frequently seeking new pages.

Although both tables and indexes have fillfactor with the same goal (accommodating UPDATE), the details differ significantly:

Tables: If a table page still has space, UPDATE can happen within that page without needing to request a new page or go to another page with free space. Moreover, due to PostgreSQL’s unique HOT feature, in-page updates don’t update indexes, which naturally slows index bloat.
Indexes: Different data rows or cross-page updates to the same row generate new index entries. Fillfactor leaves headroom in index pages, greatly reducing index split problems.

Of course, fillfactor settings are closely tied to your workload. If data is like logs — monotonically increasing with zero updates — then setting both table and index fillfactor to 100 is reasonable. But most production tables have updates, and table/index fillfactor should not be 100. For frequent UPDATE workloads, fillfactor should be set even lower.

However, PostgreSQL’s default fillfactor values are:

Table default fillfactor=100
Index default fillfactor=90

With table fillfactor=100, HOT is completely unusable! Any UPDATE immediately seeks a new data page and creates a new index entry in the index’s 10% headroom. Eventually, update-heavy workloads constantly update indexes, and even 90 fillfactor on the index can’t hold up, leading to index splits…

Here’s a fillfactor test — two tables differ only in fillfactor, updating the same amount of data, comparing the final shared hit difference:

create table tab4(a bigserial,b char(100));
create index idx_tab4_a on tab4(a);
alter index idx_tab4_a set (deduplicate_items=off); --disable index deduplication
alter table tab4 alter column b set storage PLAIN; --disable toast
alter table tab4 set (autovacuum_enabled = off); --disable autovacuum

--tab5 has the same definition as tab4, except table and index fillfactor are adjusted
alter table tab5 set (fillfactor=70);
alter index idx_tab5_a set (fillfactor=80);

insert into tab4(b) values('lllllllllll'); 
--Repeatedly update one row
DO $$
begin
	FOR i IN 1..10000 LOOP
		update tab4 set b=md5(i::text) where a=1;
	END LOOP; 
end $$;;

--Primary key query with default fillfactor
lzldb=> explain (analyze,buffers) select * from tab4 where a=1;
 QUERY PLAN 
----------------------------------------------------------------------------------------------------------------------
 Bitmap Heap Scan on tab4 (cost=4.28..53.88 rows=16 width=412) (actual time=0.894..0.895 rows=1 loops=1)
 Recheck Cond: (a = 1)
 Heap Blocks: exact=1
 Buffers: shared hit=174
 -> Bitmap Index Scan on idx_tab4_a (cost=0.00..4.28 rows=16 width=0) (actual time=0.023..0.023 rows=173 loops=1)
 Index Cond: (a = 1)
 Buffers: shared hit=1
 Planning Time: 0.057 ms
 Execution Time: 0.913 ms
(9 rows)

--Primary key query with lowered fillfactor
lzldb=> explain (analyze,buffers) select * from tab5 where a=1;
 QUERY PLAN 
-----------------------------------------------------------------------------------------------------------------------
 Bitmap Heap Scan on tab5 (cost=4.39..56.41 rows=14 width=4012) (actual time=3.367..3.369 rows=1 loops=1)
 Recheck Cond: (a = 1)
 Heap Blocks: exact=1
 Buffers: shared hit=1434
 -> Bitmap Index Scan on idx_tab5_a (cost=0.00..4.39 rows=14 width=0) (actual time=0.195..0.195 rows=1429 loops=1)
 Index Cond: (a = 1)
 Buffers: shared hit=5
 Planning Time: 0.059 ms
 Execution Time: 3.390 ms

After lowering fillfactor, the reduction in shared hits is very significant, and Execution Time improves several times over. In fact, both data pages and index pages decreased.

So, on update-heavy production tables, lowering table and index fillfactor can mitigate bloat problems.

Summary
#

Although index bloat always accompanies table bloat, their principles differ. HOT doesn’t update index entries; cross-page updates create new index entries.

Lowering table and index fillfactor can slow bloat in update-heavy production tables, ultimately also slowing down SQL queries like primary key lookups.

There are also several kernel-level features for improving index space efficiency:

Cleaning dead index entries during index scans (index tuple deletion)
Cleaning dead index entries during index splits (Bottom-Up index tuple deletion)
Vacuum marking pages of entirely dead index entries (Deleting entire pages during VACUUM)
Quickly locating recycled index pages during index splits (Placing deleted pages in the FSM)

references
#

src/backend/access/nbtree/README https://mp.weixin.qq.com/s/GBN7dFQU72BfzvLSzlLmYA pg事务：事务相关元组结构 https://www.cybertec-postgresql.com/en/killed-index-tuples/ https://www.cybertec-postgresql.com/en/index-bloat-reduced-in-postgresql-v14/?spm=a2c6h.12873639.article-detail.8.2f153438mIV8JK https://www.cybertec-postgresql.com/en/b-tree-index-improvements-in-postgresql-v12/ https://www.cybertec-postgresql.com/en/b-tree-index-deduplication/

How Does PG Access Basic System Tables Before pg_class Exists?

Mon, 12 Aug 2024 00:00:00 +0000

How does the database access system tables before pg_class exists? This question can be divided into two stages:

Database cluster initialization — at this point no database exists at all, so how to construct and access system tables like pg_class is a problem.
Private memory initialization of system tables. PG stores system table information in the local backend process. How does the backend load pg_class during initialization?

Initializing the Data Dictionary
#

When the database hasn’t been initialized yet, it’s obviously impossible to access the data dictionary to initialize objects like database, pg_class, etc., because without a database you can’t CREATE DATABASE, and without pg_class you can’t look up metadata information.

PG uses a special language in BKI files to initialize some data structures, then initializes a primitive database in bootstrap mode¹.

Compilation Phase: genbki.h & genbki.pl
#

src/include/catalog/genbki.h:

 * genbki.h defines CATALOG(), BKI_BOOTSTRAP and related macros
 * so that the catalog header files can be read by the C compiler.
 * (These same words are recognized by genbki.pl to build the BKI
 * bootstrap file from these header files.)

genbki.h is quite minimal — mainly macro definitions for catalog-related operations, as well as macros for the BKI bootstrap file. Data dictionary header files all include genbki.h.

genbki.pl reads the .h table definition files from /src/include/catalog during compilation (excluding pg_*_d.h), and creates the postgres.bki file and pg_*_d.h header files.

Taking pg_class as an example:

[postgres@catalog]$ ll |grep pg_class
-rw-r----- 1 postgres postgres 3682 Aug 6 2019 pg_class.dat
lrwxrwxrwx 1 postgres postgres 86 Apr 8 20:31 pg_class_d.h -> /lzl/soft/postgresql-11.5/src/backend/catalog/pg_class_d.h
-rw-r----- 1 postgres postgres 5219 Aug 6 2019 pg_class.h

The pg_*_d.h header files are generated by genbki.pl. All pg_*_d.h files contain the following line:

It has been GENERATED by src/backend/catalog/genbki.pl

Each data dictionary has a struct typedef struct FormData_*catalogname* for storing the row data of the data dictionary², for example pg_class’s FormData_pg_class:

CATALOG(pg_class,1259,RelationRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(83,RelationRelation_Rowtype_Id) BKI_SCHEMA_MACRO
{
	/* oid */
	Oid			oid;

	/* class name */
	NameData	relname;

	/* OID of namespace containing this class */
	Oid			relnamespace BKI_DEFAULT(pg_catalog) BKI_LOOKUP(pg_namespace);

	/* OID of entry in pg_type for relation's implicit row type, if any */
	Oid			reltype BKI_LOOKUP_OPT(pg_type);

	/* OID of entry in pg_type for underlying composite type, if any */
	Oid			reloftype BKI_DEFAULT(0) BKI_LOOKUP_OPT(pg_type);

	/* class owner */
	Oid			relowner BKI_DEFAULT(POSTGRES) BKI_LOOKUP(pg_authid);

...

	/* access-method-specific options */
	text		reloptions[1] BKI_DEFAULT(_null_);

	/* partition bound node tree */
	pg_node_tree relpartbound BKI_DEFAULT(_null_);
#endif
} FormData_pg_class;

pg_class’s OID is hardcoded as 1259, and all fields are in the FormData_pg_class struct.

After initializing the struct for user data storage, the corresponding .dat file is used to insert base data. pg_class inserts 4 rows of data, which can be understood as bootstrap items (49 data dictionary tables in PG15):

{ oid => '1247',
 relname => 'pg_type', reltype => 'pg_type' },
{ oid => '1249',
 relname => 'pg_attribute', reltype => 'pg_attribute' },
{ oid => '1255',
 relname => 'pg_proc', reltype => 'pg_proc' },
{ oid => '1259',
 relname => 'pg_class', reltype => 'pg_class' },

postgres=# select oid,relname from pg_class where oid::int >=1247 and oid::int<=1259;
 oid | relname
------+--------------
 1247 | pg_type
 1249 | pg_attribute
 1255 | pg_proc
 1259 | pg_class

Once the base data dictionary is written, everything else can be generated from it.

Database Initialization Phase: initdb & postgres.bki
#

Comment from initdb.c:

 * To create template1, we run the postgres (backend) program in bootstrap
 * mode and feed it data from the postgres.bki library file. After this
 * initial bootstrap phase, some additional stuff is created by normal
 * SQL commands fed to a standalone backend.

The backend is launched in bootstrap mode and runs the postgres.bki script. postgres.bki can execute relevant functions without any system tables. Only after this can normal SQL files and standard backend processes be used.

template1 can be called the bootstrap database. The postgres and template0 databases are created only after template1 is established:

void
initialize_data_directory(void)
{
...
	/* Bootstrap template1 */
	bootstrap_template1();
...
	make_template0(cmdfd);

	make_postgres(cmdfd);

	PG_CMD_CLOSE;

	check_ok();
}

Once template1 exists, make_template0 and make_postgres create the corresponding template0 and postgres databases, using the normal SQL CREATE DATABASE command:

/*
 * copy template1 to postgres
 */
static void
make_postgres(FILE *cmdfd)
{
	const char *const *line;

	/*
	 * Just as we did for template0, and for the same reasons, assign a fixed
	 * OID to postgres and select the file_copy strategy.
	 */
	static const char *const postgres_setup[] = {
		"CREATE DATABASE postgres OID = " CppAsString2(PostgresDbOid)
		" STRATEGY = file_copy;\n\n",
		"COMMENT ON DATABASE postgres IS 'default administrative connection database';\n\n",
		NULL
	};

	for (line = postgres_setup; *line; line++)
		PG_CMD_PUTS(*line);
}

Backend Local Cache of Data Dictionary
#

For PG private memory basics, refer to PostgreSQL Memory Analysis³.

PG’s data dictionary information is stored in the local backend process, not shared. The data dictionary cache mainly focuses on syscache/catcache and relcache, which cache system table and table schema information respectively.

syscache/catcache is used to cache system tables, with syscache acting as the upper layer of catcache. syscache is an array where each element corresponds to a catcache, and each catcache corresponds to a system table¹.

//PG15.3 SysCacheSize=35
static CatCache *SysCache[SysCacheSize];

When PG forks a backend, it calls InitPostgres, which calls the initialization functions for syscache/catcache and relcache. Let’s look at backend initialization.

syscache/catcache Initialization
#

struct cachedesc
{
	Oid			reloid;			/* OID of the relation being cached */
	Oid			indoid;			/* OID of index relation for this cache */
	int			nkeys;			/* # of keys needed for cache lookup */
	int			key[4];			/* attribute numbers of key attrs */
	int			nbuckets;		/* number of hash buckets for this cache */
};

static const struct cachedesc cacheinfo[] = {
	{
...
	{RelationRelationId,		/* RELNAMENSP */
		ClassNameNspIndexId,
		2,
		{
			Anum_pg_class_relname,
			Anum_pg_class_relnamespace,
			0,
			0
		},
		128
	},
	{RelationRelationId,		/* RELOID */
		ClassOidIndexId,
		1,
		{
			Anum_pg_class_oid,
			0,
			0,
			0
		},
		128
...
};

For example, Anum_pg_class_oid is defined in pg_class_d.h generated by genbki.pl:

#define Anum_pg_class_oid 1

reloid is the OID:

 select oid,relname from pg_class where oid::int >=1247 and oid::int<=1259;
 oid | relname
------+--------------
 1259 | pg_class

InitCatalogCache actually initializes the syscache array, i.e., initializes all catcaches. InitCatalogCache eventually fully initializes CatCache through InitCatCache (one of which is for pg_class):

void
InitCatalogCache(void)
{
...
	for (cacheId = 0; cacheId < SysCacheSize; cacheId++)
	{
		SysCache[cacheId] = InitCatCache(cacheId,
										 cacheinfo[cacheId].reloid,
										 cacheinfo[cacheId].indoid,
										 cacheinfo[cacheId].nkeys,
										 cacheinfo[cacheId].key,
										 cacheinfo[cacheId].nbuckets);
		if (!PointerIsValid(SysCache[cacheId]))
			elog(ERROR, "could not initialize cache %u (%d)",
				 cacheinfo[cacheId].reloid, cacheId);
		/* Accumulate data for OID lists, too */
		SysCacheRelationOid[SysCacheRelationOidSize++] =
			cacheinfo[cacheId].reloid;
		SysCacheSupportingRelOid[SysCacheSupportingRelOidSize++] =
			cacheinfo[cacheId].reloid;
		SysCacheSupportingRelOid[SysCacheSupportingRelOidSize++] =
			cacheinfo[cacheId].indoid;
		/* see comments for RelationInvalidatesSnapshotsOnly */
		Assert(!RelationInvalidatesSnapshotsOnly(cacheinfo[cacheId].reloid));
	}
...
	CacheInitialized = true;
}

Then we come to catcache.c.

InitCatCache allocates memory and manages it in CacheMemoryContext. It only assigns some macro-defined OIDs to the corresponding catcache — at this point, tables are not yet opened:

/*
 *		InitCatCache
 *
 *	This allocates and initializes a cache for a system catalog relation.
 *	Actually, the cache is only partially initialized to avoid opening the
 *	relation. The relation will be opened and the rest of the cache
 *	structure initialized on the first access.
 */
CatCache *
InitCatCache(int id,
			 Oid reloid,
			 Oid indexoid,
			 int nkeys,
			 const int *key,
			 int nbuckets)
{
...

	oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
...
	sz = sizeof(CatCache) + PG_CACHE_LINE_SIZE;
	cp = (CatCache *) CACHELINEALIGN(palloc0(sz));
	cp->cc_bucket = palloc0(nbuckets * sizeof(dlist_head));

	/*
	 * initialize the cache's relation information for the relation
	 * corresponding to this cache, and initialize some of the new cache's
	 * other internal fields. But don't open the relation yet.
	 */
	cp->id = id;
	cp->cc_relname = "(not known yet)";
	cp->cc_reloid = reloid;
	cp->cc_indexoid = indexoid;
	cp->cc_relisshared = false; /* temporary */
	cp->cc_tupdesc = (TupleDesc) NULL;
	cp->cc_ntup = 0;
	cp->cc_nbuckets = nbuckets;
	cp->cc_nkeys = nkeys;
	for (i = 0; i < nkeys; ++i)
		cp->cc_keyno[i] = key[i];
...
	MemoryContextSwitchTo(oldcxt);

	return cp;
}

id is the index of the catcache array element. The assigned reloid is the known OID from cacheinfo, and key[4] from cacheinfo is also assigned. Other information is mostly unknown yet — for example, relname, tupdesc — because system tables haven’t been opened yet.

catcache only opens tables during search operations. Although the function name contains *init*, it’s no longer in the initialization process — the relevant functions won’t be shown here.

After syscache/catcache initialization completes, there is actually no tuple information at all.

relcache Initialization
#

The relcache initialization is well explained in PostgreSQL Memory Analysis.

relcache initialization has 5 phases:

RelationCacheInitialize - initializes relcache, initially empty
RelationCacheInitializePhase2 - initializes shared catalogs and loads 5 global system tables
RelationCacheInitializePhase3 - completes relcache initialization and loads 4 basic system tables
RelationIdGetRelation - gets rel description by relation id
RelationClose - closes a relation

Both RelationCacheInitializePhase2 and RelationCacheInitializePhase3 load system tables, and they must be in order.

RelationCacheInitializePhase2 loads several system tables — interested readers can check the function themselves. RelationCacheInitializePhase3 is the one relevant to our question, let’s look at that:

/*
 *		RelationCacheInitializePhase3
 *
 *		This is called as soon as the catcache and transaction system
 *		are functional and we have determined MyDatabaseId. At this point
 *		we can actually read data from the database's system catalogs.
 *		We first try to read pre-computed relcache entries from the local
 *		relcache init file. If that's missing or broken, make phony entries
 *		for the minimum set of nailed-in-cache relations. Then (unless
 *		bootstrapping) make sure we have entries for the critical system
 *		indexes. Once we've done all this, we have enough infrastructure to
 *		open any system catalog or use any catcache. The last step is to
 *		rewrite the cache files if needed.
 */
void
RelationCacheInitializePhase3(void)
{
...
	if (IsBootstrapProcessingMode() ||
		!load_relcache_init_file(false))
	{
		needNewCacheFile = true;

		formrdesc("pg_class", RelationRelation_Rowtype_Id, false,
				 Natts_pg_class, Desc_pg_class);
		formrdesc("pg_attribute", AttributeRelation_Rowtype_Id, false,
				 Natts_pg_attribute, Desc_pg_attribute);
		formrdesc("pg_proc", ProcedureRelation_Rowtype_Id, false,
				 Natts_pg_proc, Desc_pg_proc);
		formrdesc("pg_type", TypeRelation_Rowtype_Id, false,
				 Natts_pg_type, Desc_pg_type);

#define NUM_CRITICAL_LOCAL_RELS 4	/* fix if you change list above */
	}

	MemoryContextSwitchTo(oldcxt);

	/* In bootstrap mode, the faked-up formrdesc info is all we'll have */
	if (IsBootstrapProcessingMode())
		return;

...
		/* now write the files */
		write_relcache_init_file(true);
		write_relcache_init_file(false);
	}
}

IsBootstrapProcessingMode is specifically designed for bootstrap mode — normal backends don’t satisfy this condition.

load_relcache_init_file(false) attempts to load system table information from the init file. load_relcache_init_file(false) passes false meaning it’s a private init file, not a shared one:

[postgres@16384]$ pwd
/pgdata/lzl/data15_6879/base/16384
-- Rough view. strings ignores some info, but table and column names are visible
[postgres@16384]$ strings pg_internal.init |grep pg_class
pg_class_oid_index
pg_class
pg_class_relname_nsp_index
[postgres@16384]$ strings pg_internal.init |grep -E "pg_class|relname"
pg_class_oid_index
pg_class
relname
relnamespace
pg_class_relname_nsp_index
relname
relnamespace

If the init file is damaged or doesn’t exist, loading the init file fails and enters the branch to load 4 basic system tables:

	// Similar to phase 2, load more system table descriptions
	if (IsBootstrapProcessingMode() ||
		!load_relcache_init_file(false))
	{
		needNewCacheFile = true;

		formrdesc("pg_class", RelationRelation_Rowtype_Id, false,
				 Natts_pg_class, Desc_pg_class);
		formrdesc("pg_attribute", AttributeRelation_Rowtype_Id, false,
				 Natts_pg_attribute, Desc_pg_attribute);
		formrdesc("pg_proc", ProcedureRelation_Rowtype_Id, false,
				 Natts_pg_proc, Desc_pg_proc);
		formrdesc("pg_type", TypeRelation_Rowtype_Id, false,
				 Natts_pg_type, Desc_pg_type);

With the 4 basic tables including pg_class, loading subsequent system table information becomes straightforward.

References
#

《PostgreSQL Kernel Analysis》 Chapters 2, 3 ↩︎ ↩︎
https://www.postgresql.org/docs/current/system-catalog-declarations.html ↩︎
PostgreSQL Memory Analysis ↩︎

When Does VACUUM Truncate Empty Pages at the End of a Table?

Mon, 12 Aug 2024 00:00:00 +0000

VACUUM Truncate
#

TRUNCATE—Specifies that VACUUM should attempt to truncate off any empty pages at the end of the table and allow the disk space for the truncated pages to be returned to the operating system. This is normally the desired behavior and is the default unless the vacuum_truncate option has been set to false for the table to be vacuumed. Setting this option to false may be useful to avoid ACCESS EXCLUSIVE lock on the table that the truncation requires. This option is ignored if the FULL option is used.

AKA, the truncate option in VACUUM is enabled by default. It removes empty pages at the end of the table, acquiring an AccessExclusiveLock (level 8 lock) on the table during the operation.

Today I found that in a certain environment, after deleting all data with DELETE FROM, neither autovacuum nor manual VACUUM reclaimed the space.

Reproducing the issue:

create table lzl1(a int);
insert into lzl1 select generate_series(1,1000) a;
analyze lzl1;

lzldb=# select relname,relpages,reltuples from pg_class where relname='lzl1';
 relname | relpages | reltuples
---------+----------+-----------
 lzl1 | 5 | 1000

relpages is 5, so the last page number is 4.

lzldb=> select t_ctid,lp,case lp_flags when 0 then '0:LP_UNUSED' when 1 then 'LP_NORMAL' when 2 then 'LP_REDIRECT' when 3 then 'LP_DEAD' end as lp_flags,t_xmin,t_xmax,t_field3 as t_cid, raw_flags, info.combined_flags,substring(t_data,0,40) from heap_page_items(get_raw_page('lzl1',4)) item,LATERAL heap_tuple_infomask_flags(t_infomask, t_infomask2) info order by lp;
 t_ctid | lp | lp_flags | t_xmin | t_xmax | t_cid | raw_flags | combined_flags | substring
--------+----+-----------+--------+--------+-------+-----------------------------------------+----------------+------------
 (4,1) | 1 | LP_NORMAL | 772 | 0 | 0 | {HEAP_XMIN_COMMITTED,HEAP_XMAX_INVALID} | {} | \x89030000
 (4,2) | 2 | LP_NORMAL | 772 | 0 | 0 | {HEAP_XMIN_COMMITTED,HEAP_XMAX_INVALID} | {} | \x8a030000
 ...

delete from lzl1;
vacuum lzl1;

lzldb=> select t_ctid,lp,case lp_flags when 0 then '0:LP_UNUSED' when 1 then 'LP_NORMAL' when 2 then 'LP_REDIRECT' when 3 then 'LP_DEAD' end as lp_flags,t_xmin,t_xmax,t_field3 as t_cid, raw_flags, info.combined_flags,substring(t_data,0,40) from heap_page_items(get_raw_page('lzl1',4)) item,LATERAL heap_tuple_infomask_flags(t_infomask, t_infomask2) info order by lp;
 t_ctid | lp | lp_flags | t_xmin | t_xmax | t_cid | raw_flags | combined_flags | substring
--------+----+-------------+--------+--------+-------+-----------+----------------+-----------
 | 1 | 0:LP_UNUSED | | | | | |
 lzldb=# select relname,relpages,reltuples from pg_class where relname='lzl1';
 relname | relpages | reltuples
---------+----------+-----------
 lzl2 | 5 | 0

It looks like all dead tuples were reclaimed, but the space is still occupied — the pages were not freed. Why doesn’t it truncate when the table is completely empty? Let’s dig into this question.

Source Code Analysis of `should_attempt_truncation`
#

(Unless otherwise noted, the version referenced is PG 11.)

In vacuumlazy.c there’s a pithily named function should_attempt_truncation — this is the function that decides whether truncation is needed:

static bool
should_attempt_truncation(LVRelStats *vacrelstats)
{
	BlockNumber possibly_freeable;

	possibly_freeable = vacrelstats->rel_pages - vacrelstats->nonempty_pages;
	if (possibly_freeable > 0 &&
		(possibly_freeable >= REL_TRUNCATE_MINIMUM ||
		 possibly_freeable >= vacrelstats->rel_pages / REL_TRUNCATE_FRACTION) &&
		old_snapshot_threshold < 0)
		return true;
	else
		return false;
}

Where:

#define REL_TRUNCATE_MINIMUM 1000
#define REL_TRUNCATE_FRACTION 16

So the conditions for truncation are:

Number of empty trailing pages > 1000, or number of empty trailing pages > 1/16 of total pages
old_snapshot_threshold < 0

The first rule exists to avoid constantly truncating tiny bits of trailing empty pages — reclaiming that negligible space isn’t worth the time and the AccessExclusiveLock. It’s unnecessary.

The second rule is explained as follows:

 * Also don't attempt it if we are doing early pruning/vacuuming, because a
 * scan which cannot find a truncated heap page cannot determine that the
 * snapshot is too old to read that page. We might be able to get away with
 * truncating all except one of the pages, setting its LSN to (at least) the
 * maximum of the truncated range if we also treated an index leaf tuple
 * pointing to a missing heap page as something to trigger the "snapshot too
 * old" error, but that seems fragile and seems like it deserves its own patch
 * if we consider it.

“Because VACUUM scanning cannot yet confirm whether page data has snapshot-too-old issues, and there are LSN and index page complications, the code logic looks fiddly. If this feature is needed, a dedicated patch would be required.”

OK, so it looks like the code simply doesn’t check whether a page actually has snapshot-too-old issues. It takes the blunt approach of checking old_snapshot_threshold < 0 — the database itself must have snapshot-too-old disabled before truncation is attempted.

Going back to the earlier problem where VACUUM didn’t reclaim space: since DELETE removed all data, the condition “empty trailing pages > 1/16 of total pages” was definitely satisfied. However, old_snapshot_threshold was actually enabled in that environment:

lzldb=> show old_snapshot_threshold ;
 old_snapshot_threshold
------------------------
 1h

Disabling old_snapshot_threshold and then doing the delete-all + VACUUM will reclaim the space. Disabling old_snapshot_threshold requires a database restart.

-- After restart
lzldb=> show old_snapshot_threshold ;
 old_snapshot_threshold
------------------------
 -1
lzldb=> select pg_relation_filepath('lzl1');
 pg_relation_filepath
----------------------
 base/16384/16446

lzldb=> vacuum lzl1;
-- Pages successfully reclaimed
lzldb=# select relname,relpages,reltuples from pg_class where relname='lzl1';
 relname | relpages | reltuples
---------+----------+-----------
 lzl1 | 0 | 0
-- Table not rebuilt
lzldb=> select pg_relation_filepath('lzl1');
 pg_relation_filepath
----------------------
 base/16384/16446

All pages successfully reclaimed, table not rebuilt. Problem located.

But to understand the VACUUM truncation mechanism more deeply, let’s continue to the next section.

Source Code Analysis of `lazy_truncate_heap`
#

Relying solely on should_attempt_truncation to judge truncation isn’t rigorous enough. We also need to look at lazy_truncate_heap, the function that actually performs truncation, which has additional checks:

/*
 * lazy_truncate_heap - try to truncate off any empty pages at the end
 */
static void
lazy_truncate_heap(Relation onerel, LVRelStats *vacrelstats)
{
	BlockNumber old_rel_pages = vacrelstats->rel_pages;
	BlockNumber new_rel_pages;
	int			lock_retry;

	/* Report that we are now truncating */
	pgstat_progress_update_param(PROGRESS_VACUUM_PHASE,
								 PROGRESS_VACUUM_PHASE_TRUNCATE);

	/*
	 * Loop until no more truncating can be done.
	 */
	do
	{
		PGRUsage	ru0;

		pg_rusage_init(&ru0);

		/*
		 * We need full exclusive lock on the relation in order to do
		 * truncation. If we can't get it, give up rather than waiting --- we
		 * don't want to block other backends, and we don't want to deadlock
		 * (which is quite possible considering we already hold a lower-grade
		 * lock).
		 */
		Vacrelstats->lock_waiter_detected = false;
		lock_retry = 0;
		while (true)
		{
			// If we can acquire the lock, break out of while
			if (ConditionalLockRelation(onerel, AccessExclusiveLock))
				break;

			/*
			 * Check for interrupts while trying to (re-)acquire the exclusive
			 * lock.
			 */
			CHECK_FOR_INTERRUPTS();

			// If lock not immediately acquired, initially (++lock_retry)=1, <=100;
			// when >100, give up truncation and return
			if (++lock_retry > (VACUUM_TRUNCATE_LOCK_TIMEOUT /
								VACUUM_TRUNCATE_LOCK_WAIT_INTERVAL))
			{
				/*
				 * We failed to establish the lock in the specified number of
				 * retries. This means we give up truncating.
				 */
				Vacrelstats->lock_waiter_detected = true;
				ereport(elevel,
						(errmsg("\"%s\": stopping truncate due to conflicting lock request",
								RelationGetRelationName(onerel))));
				return;
			}

			// Sleep 50ms. Looks a bit crude. Theoretical max wait: 50*100=5s
			pg_usleep(VACUUM_TRUNCATE_LOCK_WAIT_INTERVAL * 1000L);
		}

		// After acquiring the exclusive lock, check if new tuples arrived during VACUUM.
		// If so, don't truncate.
		new_rel_pages = RelationGetNumberOfBlocks(onerel);
		if (new_rel_pages != old_rel_pages)
		{
			UnlockRelation(onerel, AccessExclusiveLock);
			return;
		}

		new_rel_pages = count_nondeletable_pages(onerel, vacrelstats);
		// If new tuples were written during VACUUM, don't truncate
		if (new_rel_pages >= old_rel_pages)
		{
			/* can't do anything after all */
			UnlockRelation(onerel, AccessExclusiveLock);
			return;
		}

		/*
		 * Okay to truncate.
		 */
		RelationTruncate(onerel, new_rel_pages);

		// Release lock immediately after truncation
		UnlockRelation(onerel, AccessExclusiveLock);

...
	} while (new_rel_pages > vacrelstats->nonempty_pages &&
			 vacrelstats->lock_waiter_detected);
}

Where:

#define VACUUM_TRUNCATE_LOCK_WAIT_INTERVAL 50 /* microseconds!! */
#define VACUUM_TRUNCATE_LOCK_TIMEOUT 5000 /* microseconds!! */

The main function actually called is RelationTruncate. The bulk of the preceding code is all about trying to acquire the AccessExclusiveLock. Beyond the two conditions mentioned earlier, truncation also won’t happen in these two cases:

Failed to acquire AccessExclusiveLock
New data was written during the VACUUM

VACUUM Truncate May Wait Up to 5 Seconds
#

While reading the lazy_truncate_heap source code above, I noticed the lock acquisition retry loop has a somewhat crude wait:

pg_usleep(VACUUM_TRUNCATE_LOCK_WAIT_INTERVAL * 1000L);

Each loop iteration sleeps 50ms. The theoretical maximum wait is 50×100 = 5 seconds!

Let’s test this wait time:

Window 1	Window 2
create table lzl2;
alter table lzl2 set (autovacuum_enabled=off);;
insert into lzl2 select generate_series(1,1000) a;
delete from lzl2;
begin;
select * from lzl2;
	\timing
	vacuum lzl2; – Time: 5022.122 ms (00:05.022)

We can see the wait time is about 5 seconds.

If you’re fast enough, you can open a third window and grab a pstack of session 2:

[postgres@cncq081298 lzl]$ pstack 4113
#0 0x00002b92a978c013 in __select_nocancel () from /lib64/libc.so.6
#1 0x000000000086225a in pg_usleep (microsec=microsec@entry=50000) at pgsleep.c:56
#2 0x00000000005e8212 in lazy_truncate_heap (vacrelstats=0xfc4490, onerel=0x2b92a8bc88d8) at vacuumlazy.c:1861
#3 lazy_vacuum_rel (onerel=onerel@entry=0x2b92a8bc88d8, options=options@entry=5, params=params@entry=0x7ffc96bb31d0, bstrategy=<optimized out>) at vacuumlazy.c:290
#4 0x00000000005e4551 in vacuum_rel (relid=32778, relation=<optimized out>, options=options@entry=5, params=params@entry=0x7ffc96bb31d0) at vacuum.c:1572
#5 0x00000000005e55ac in vacuum (options=5, relations=0xfc6540, params=params@entry=0x7ffc96bb31d0, bstrategy=<optimized out>, bstrategy@entry=0x0, isTopLevel=isTopLevel@entry=true) at vacuum.c:340
...

It reached pg_usleep inside lazy_truncate_heap, passing entry=50000 microsec. In reality, pg_usleep looped 100 times, total wait time 50000×100 microseconds = 5 seconds.

Later, in PG 15, this code was improved by replacing pg_usleep with WaitLatch:

(void) WaitLatch(MyLatch,
WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
VACUUM_TRUNCATE_LOCK_WAIT_INTERVAL,
WAIT_EVENT_VACUUM_TRUNCATE);
ResetLatch(MyLatch);

VACUUM Truncate Summary
#

Conditions for VACUUM to trigger truncation (all must be met):

Empty trailing pages > 1000, or empty trailing pages > 1/16 of total pages
old_snapshot_threshold < 0
Before PG 15 (exclusive): must acquire AccessExclusiveLock within 5 seconds
No new data written during the VACUUM

This article was originally published in Chinese on lastdba.com.

PostgreSQL源码解析 on Last DBA

Parameters on the Control File and Primary-Standby Parameter Mismatch Issues

PARAMETER_CHANGE and Database Parameters on the Control File #

wal_log_hints Primary-Standby Mismatch #

Additional Knowledge: What is XLOG_RUNNING_XACTS #

track_commit_timestamp Primary-Standby Mismatch #

PG14+ Pause Recovery #

Summary of the 8 Parameters #

A Brief Analysis of PostgreSQL TRUNCATE

Command Options #

MVCC / Transaction #

Feature Updates #

Functional Differences Between pg TRUNCATE and Other Databases #

What TRUNCATE Does #

Source Code Analysis #

Reference #

From Extremely Slow Unique Index Scan to Index Bloat

How Did a Primary Key Query Access Multiple Data Pages? #

kill index item #

Bottom-Up deletion #

index deduplication #

Can unique indexes produce deduplication? #

Disabling deduplication #

What does VACUUM do? #

Placing deleted pages in the FSM #

Flowchart: Index Bloat and Cleanup #

fillfactor #

Summary #

references #

How Does PG Access Basic System Tables Before pg_class Exists?

Initializing the Data Dictionary #

Compilation Phase: genbki.h & genbki.pl #

Database Initialization Phase: initdb & postgres.bki #

Backend Local Cache of Data Dictionary #

syscache/catcache Initialization #

relcache Initialization #

References #

When Does VACUUM Truncate Empty Pages at the End of a Table?

VACUUM Truncate #

Source Code Analysis of should_attempt_truncation #

Source Code Analysis of lazy_truncate_heap #

VACUUM Truncate May Wait Up to 5 Seconds #

VACUUM Truncate Summary #

PARAMETER_CHANGE and Database Parameters on the Control File
#

wal_log_hints Primary-Standby Mismatch
#

Additional Knowledge: What is XLOG_RUNNING_XACTS
#

track_commit_timestamp Primary-Standby Mismatch
#

PG14+ Pause Recovery
#

Summary of the 8 Parameters
#

Command Options
#

MVCC / Transaction
#

Feature Updates
#

Functional Differences Between pg TRUNCATE and Other Databases
#

What TRUNCATE Does
#

Source Code Analysis
#

Reference
#

How Did a Primary Key Query Access Multiple Data Pages?
#

kill index item
#

Bottom-Up deletion
#

index deduplication
#

Can unique indexes produce deduplication?
#

Disabling deduplication
#

What does VACUUM do?
#

Placing deleted pages in the FSM
#

Flowchart: Index Bloat and Cleanup
#

fillfactor
#

Summary
#

references
#

Initializing the Data Dictionary
#

Compilation Phase: genbki.h & genbki.pl
#

Database Initialization Phase: initdb & postgres.bki
#

Backend Local Cache of Data Dictionary
#

syscache/catcache Initialization
#

relcache Initialization
#

References
#

VACUUM Truncate
#

Source Code Analysis of `should_attempt_truncation`
#

Source Code Analysis of `lazy_truncate_heap`
#

VACUUM Truncate May Wait Up to 5 Seconds
#

VACUUM Truncate Summary
#