杂项 on Last DBA

A DBA's Perspective on the 0526 Approved Database List

Fri, 29 May 2026 00:00:00 +0000

AI rate 5%

TL;DR
#

On May 26, the Xinchuang Database List 2026 No. 2 was released, with 23 products passing (8 centralized + 15 distributed) — the most ever. Most notably: Ping An, UnionPay, China Mobile, and China Telecom — four major buyers — had their self-incubated databases debut on the list. The Xinchuang logic has changed — buyers are no longer just buyers.

The Latest List
#

Historical batch statistics for the Xinchuang database list. Data source: China Information Security Evaluation Center (itsec.gov.cn), 8 batches total, 4 containing databases.

By Batch

Batch	Date	Database Products	Achieved Level II
2023#1	2023-12-26	11 (centralized)	None
2024#2	2024-09-30	17 (6 centralized + 11 distributed)	GaussDB
2025#2	2025-08-22	3 (centralized)	None
2026#2	2026-05-26	23 (8 centralized + 15 distributed)	Dameng/Yashan/GaussDB/GoldenDB

By Appearances (≥2 times)

Vendor	Count
Dameng	3
GBASE	3
Alibaba Cloud	3
HighGo	2
Tencent Cloud	2
East Golden	2
Vastdata	2
Huawei Cloud	2
ZTE (GoldenDB)	2
OceanBase	2
Kingbase	2
Shentong	2
Xugu	2
Yashan	2
Only 1 time	PingCAP/Wanli/Uxin/Ping An/China Mobile/UnionPay/Telecom Cloud/Timecho/Transwarp/DolphinDB/Z-Range/CM Suzhou

By Category: Big Tech / Unicorn / Major Buyer

Category	Vendors
Big Tech	Huawei Cloud (GaussDB/TaurusDB/DWS), Alibaba Cloud (PolarDB/AnalyticDB), Tencent Cloud (TDSQL), ZTE (GoldenDB), OceanBase (Ant Group)
Unicorns	PingCAP (TiDB), Yashan (SICS), Transwarp (ArgoDB), Timecho (TimechoDB), DolphinDB
Major Buyers	Ping An Tech (RASESQL), China UnionPay (UPDRDB), China Mobile (Panwei + He3DB), China Telecom Cloud (TeleDB)
Traditional Xinchuang	Dameng, Kingbase, GBASE, Shentong, HighGo, Xugu, Vastdata, East Golden, Wanli, Uxin

The Floodgates Open
#

When this list came out, my reaction was four words: the floodgates opened. 23 products — the most ever. A few highlights:

Ping An RASESQL. The most unexpected. Ping An Group’s fintech capabilities have always been strong, but there was almost no public information about them building a database. Seeing “RASESQL” on the list stunned me for several seconds. A financial buyer of Ping An’s scale — once their self-developed database passes national testing, their internal Xinchuang replacement roadmap gains one more path.

UnionPay UPDRDB. Equally mysterious. I had no idea UnionPay was building a distributed database before this. UnionPay’s transaction volume speaks for itself — a distributed database that can handle their own business won’t be technically weak.

Alibaba Cloud PolarDB for MySQL. The MySQL-compatible edition of PolarDB not passing had been something many people remembered. Now, all three of PolarDB’s main lines — PG edition, distributed edition, MySQL edition — have passed. Add AnalyticDB, and Alibaba Cloud’s database family is basically complete.

China Mobile Panwei + China Telecom TeleDB. China Mobile already had He3DB (CM Suzhou) pass national testing last year; this year Panwei is their second product. China Telecom TeleDB debuts. Both telecom operators now have their own incubated Xinchuang databases, which should significantly reduce their respective Xinchuang replacement pressure. Interestingly, China Unicom has been silent — their Xinchuang strategy is clearly different from Mobile and Telecom.

Transwarp ArgoDB. Transwarp started in the big data/Hadoop ecosystem and now their distributed database has passed national testing. Once crowned “China’s First Domestic Big Data Infrastructure Software Stock” with a market cap exceeding 30 billion, their path from data lake to Xinchuang database has been validated.

Impact
#

The most important signal from this floodgate opening: buyers can self-develop databases.

What are the implications?

Major buyers who succeed at self-development don’t have to be lambs to the slaughter.
Those major buyers who haven’t built one yet may restart their self-development efforts.
The market share that big tech and unicorns could compete for in the domestic database market just shrank.

Financial industry players UnionPay and Ping An, telecom players China Mobile and China Telecom — all passed national testing, effectively earning a “R&D Success” gold badge. Internally, each organization must be celebrating. For external vendors, what they’ve lost isn’t just major clients — more precisely, they’ve lost absolute bargaining power.

“I know you’re in a tough spot, and I know you can’t afford not to buy, so I’ll swap the butcher’s knife for a dragon-slaying blade and slaughter you to death” — for buyers who successfully incubated their own databases, this kind of predicament has been substantially eased. That’s significant.

As for where Xinchuang policy goes next, nobody can say. Based on previous lists, things should be getting stricter (last time only 3 databases passed), but this time they unexpectedly opened the floodgates. A sharp contraction next round isn’t impossible. Not just China Unicom — insurance industry players like CPIC and PICC, and even capable financial institutions, could consider jumping in to hand-roll their own database.

Bittersweet Reflections
#

Since our kernel team sits right behind me, I have some understanding of the Xinchuang R&D process. After consecutive failed submissions, the entire team’s morale was extremely low. I believe we weren’t the only ones — many teams whose submissions failed felt the same. For industries like finance and telecom, there’s a Xinchuang mandate, but if your self-developed product doesn’t pass approval, there’s no choice at the corporate strategy level, and at the team level, there’s no reason for existence. That’s why “passing national testing” carries such weight and influence. Thankfully they passed — heartfelt congratulations to them! RaseSQL No.1!

At the same time, it’s clear that Xinchuang results and direction are unstable, volatile, and impactful. It determines some companies’ strategies and many people’s fates. I myself am even a piece on this wheel of fortune.

Beyond those on the list, many organizations poured enormous effort but remain off the list. Their products might be terrible, or they might be excellent. But national testing is that stark watershed — a mysterious ticket of admission. Pass or fail — in the domestic market, those are two entirely different concepts.

OK, just some thoughts — might delete later.

Reference
#

https://www.itsec.gov.cn/aqkkcp/cpgg/

Original link: https://lastdba.com/2026/05/29/xinchuang-db-2026-review/

My Blog is Live

Sat, 16 May 2026 00:00:00 +0000

It’s Live!
#

The blog is finally live.

URL: https://lastdba.com

Accessible from China, mobile-friendly too.

76 articles — all PostgreSQL writing from the past few years: case studies, internals, source code analysis, paper deep reads.

This is a proper launch: new framework, new domain, new theme — rebuilt from the ground up.

Highlights
#

Clean Interface

Minimalist, reader-friendly design with a useful search feature.

Framework: Jekyll → Hugo

Version 1: Jekyll + minima theme + 2000 lines of CSS

Version 2: Hugo + Blowfish theme + 0 lines of CSS

V1 was decent, but building the UI myself was exhausting. I remembered vonng had written an article about website architecture choices, so I just went and borrowed from it. I explained the architecture to AI and had it learn from vonng.com — the page quality jumped up a level instantly. A few more tweaks and it was done.

Domain: github.io → lastdba.com

Bought lastdba.com, configured Cloudflare. GitHub Pages with custom domain, free HTTPS certificate, auto-renewal. Now accessible without VPN!

Image Localization

Previously, article images were scattered everywhere — CSDN CDN, GitHub PicBed, Modb OSS. CSDN has hotlink protection. GitHub PicBed on foreign networks often failed to load domestically. This time I had AI consolidate everything to local paths. No more worrying about image hosts going down. Cross-network image loading problems solved — very good.

Reflections on Going Live
#

I’d actually set up a blog URL before — just fork a blog project and deploy via GitHub Pages. The domain was liuzhilong62.github.io/blogs. But being somewhat of a quality freak (not really), the results were mediocre so I took it down. Later I just used the GitHub repo as my blog, without even enabling Pages. Recently, with more free time for various reasons, I revisited this and used Hermes to build the blog from scratch.

As a DBA and backend engineer, I know nothing about frontend stuff like Jekyll, Hugo, Blowfish, CSS. I just give Hermes a target and it does the work. When it explains things to me I don’t understand (and I’m too embarrassed to admit it), I basically just say “keep going.” I check the result in the browser — if I’m satisfied, great; occasionally I say “revert this.”

Honestly, my biggest takeaway from switching to Hugo wasn’t technical — it was “don’t reinvent the wheel.” I’d spent so much time hand-coding dark mode, TOC, search, only to discover a theme swap includes it all, and theirs looks better than mine.

Also, after hooking up lastdba.com, the blog suddenly felt “official.” liuzhilong62.github.io/blogs felt like a personal experiment; now it feels like a real website. Same content, different feeling.

What It Cost
#

All expenses:

Item	Cost
`lastdba.com` domain (Cloudflare, 1 year)	¥70
GitHub Pages hosting	¥0
Hugo framework	¥0
Blowfish theme	¥0
Cloudflare DNS + CDN	¥0
Tokens	¥60
Total	¥130

Possibly the most cost-effective personal website solution out there.

Finally
#

Some details may not be polished — feedback, bug reports, and optimization suggestions welcome.

I’ll likely keep updating.

Reference
#

https://vonng.com/

Original link: https://lastdba.com/2026/05/16/个人博客上线/

My 2025 Year-End Summary

Sun, 21 Dec 2025 00:00:00 +0000

As a DBA
#

As a DBA, I strongly believe in first principles and information theory when it comes to problem analysis. A DBA needs to deeply understand the system, understand PostgreSQL, to explain anomalies from first principles. For example, in the first half of the year I spent considerable effort understanding Linux memory, exploring the essence of memory issues and their solutions. At the same time, this year I took a step forward in system operations — no longer focusing solely on technical problems and handling, but more on providing solutions. These should encompass thinking across the PostgreSQL database technology dimension, the system dimension, and the management dimension.

Here’s a simple classification of cloud DBA work:

Many Ops papers only talk about incident handling, but in reality, incident handling probably accounts for less than 5% of actual operational workload. And whether in academia or practice, anomaly ops itself isn’t very effective anyway. So I’m not very bullish on AIOps being able to significantly help DBAs. Note that DBAs using AIOps and DBAs using AI are two different things.

Actually, this diagram is just so-so, because it doesn’t include leadership tasks, which are definitely the bulk.

Looking back at the 2023 and 2024 year-end summaries, I can simply summarize my DBA work year by year:

2023: Comprehensive PostgreSQL learning
2024: Comprehensive PostgreSQL operations
2025: Responsible for 1510 emotional value

What’s deeply ironic is that last year’s conclusion — “DBAs are providing 1510 emotional value to their leaders” — became my lived reality this year. I don’t want to say more about it. In short, it’s been exhausting, mentally draining. I hope next year brings improvement.

READING
#

This year I read even more books than last year (from 20+ to 30+), but wrote even fewer reading notes. Writing is indeed troublesome and energy-consuming, and I’ve grown to prefer the feeling of reading itself. Compared to last year, this year’s reading shows a clear decrease in PostgreSQL technical books, an increase in comprehensive technical books, and I even started reading psychology, economics, and philosophy. In short, broader hunting grounds, not limited to databases alone. Also fewer novels — novels are like snacks, and I’m increasingly losing interest in such non-nutritious content.

This year’s book list generally falls into: IT Systems, Economics, Popular Science, Spiritual, and Fiction categories. As with last year, ranked by personal preference.

IT Systems Book List:

“SRE: Google’s Approach to Service Reliability” — DBAs are not SREs, but their work involves system stability objectives, which has similarities with DBA work. Some content in this book about cloud environments or management aspects was truly enlightening — for example, SLA, systems engineering, operational pressure, busy work, role rotation, “trust the team rather than a single technical expert,” and more. Absolutely brilliant. Recently I also heard the term DBRE — Database Reliability Engineer — which fits my current role even better than DBA. In short, an excellent book, a must-read for modern ops.
“Running Linux Kernel: Introduction” — operating open-source databases requires understanding the operating system. One of my books for studying Linux memory.
“Deep Understanding of Linux Processes and Memory” — one of my books for studying Linux memory.
“Understanding the Linux Kernel” — one of my books for studying Linux memory.
“Observability Engineering” — the patterns and flaws of traditional monitoring and traditional ops, and what observability essentially means. Quite helpful.

Economics Book List:

“Microeconomics” — a masterpiece, by Daron Acemoglu. I consider it essential reading for life. This book has my best notes of any book. Not only understanding economics, but further understanding society. Some viewpoints left a deep impression on me:
- Proves why the market is an invisible hand that maximizes social surplus value — any intervention reduces social surplus value.
- Under what circumstances markets are ineffective: externalities, public resources, and common-pool resources.
- Women earn less than men in the workplace partly because women bear children and cannot participate in production during that time.
- The function of academic credentials is signaling — to a certain degree, they certify the productive value of the person.
- Business entry and exit are normal market signals, not signs of disorder.
- The trade-off between equity and efficiency is a subject of study.
“Why Nations Fail” — a masterpiece, by Daron Acemoglu. This book can be summarized in one sentence: Why do nations succeed? Because of creative destruction. Daron Acemoglu won the 2024 Nobel Prize in Economics for “research on how institutions are formed and how they affect prosperity.” What’s even more remarkable is that this book is easier to understand than other economics works. The top-recommended economics masterpiece.
“The Rational Optimist” — said to rival “Sapiens,” but it’s definitely a notch below. However, the content quality isn’t bad, and it’s more economics-oriented. Some viewpoints are very fresh, for example:
- Modern economics makes the rich richer, but the poor are not getting poorer.
- Self-sufficiency is poverty.
- What distinguishes humans from animals is barter exchange (in “Sapiens” it’s the cognitive revolution).
- Higher income leads to greater happiness — this is a fact.
- The elevation of trade in social status came from the rise of maritime trade, because land trade was unstable and easily plundered.
“Reminiscences of a Stock Operator” — feels like I learned something and nothing at the same time. Decent read though.
“Game Theory” — honestly, I found it average. Not much content, quite superficial. I mainly read it because economics books keep mentioning game theory, so I flipped through it to evaluate.
“The Wealth of Nations” — extremely dense, not for normal people to read. Incredibly content-rich. Adam Smith must have been a genius — hard to imagine what kind of mind produced this. Too difficult for me, didn’t finish, gave up.

Popular Science Book List:

“A Brief History of Intelligence” — a masterpiece, essential reading for the AI era. This book is worn from my constant reading, covered in notes everywhere. Deconstructing the human brain, understanding what intelligence is, understanding how AI came to be. I give it full marks! Now whenever I see any animal, I first think about what intelligence level it’s at…
“On Top of Tides” — by Wu Jun. Every IT professional should read this book. It tells the rise and fall of major IT companies. You can learn about Oracle, Google, Fairchild, Bell Labs, and even basics about venture capital. Every company has its own DNA, which is nearly unchangeable and determines the company’s culture and characteristics. A programmer’s must-read.
“The Almanack of Naval Ravikant” — has many useful perspectives, like views on marginal utility. And more importantly, it recommended one of my favorite books this year — “Microeconomics.” It also recommended meditation, which changed my habits.
“How to Manage a Software Company” — by Frank Slootman, a legendary Silicon Valley CEO who led three software companies (ServiceNow, Data Domain, Snowflake) to successful IPOs. A very good book, looking at company development, employee management, execution, decision-making, and decision failures from an IT company manager’s perspective. Highly recommended.
“The Economics of Aging” — by Kenichi Ohmae. Using Japan’s aging problem to glimpse China’s aging problems and opportunities. The demographic structural risks in our country are severe and about to come to a head. In this era, highly recommended reading.
“The Fourth Wave” — by Kenichi Ohmae. Mainly about how Japan missed the IT technology wave, still relying on old industries to support the national economy, appearing somewhat envious of South Korea and China. I personally love the author’s attitude of directly criticizing the prime minister, haha.
“The Checklist Manifesto” — explains the necessity of checklist inspections before Western surgical procedures. Seemingly simple steps can dramatically increase surgical success rates. This book had a big impact on my work — I genuinely brought the checklist concept into my work. I treat database operations like a surgical procedure — checklists are a simple yet necessary means to improve success rates.
“The Mythical Man-Month” — “adding people” cannot linearly reduce systems engineering project timelines, but you also can’t simply reject “adding people” because large systems engineering projects genuinely require many people collaborating. It’s a good book, but calling it a programmer’s must-read feels like a stretch.
“The Beauty of Mathematics” — by Wu Jun. Also quite good. Technology always has its mathematical foundations. This book accessibly tells the beauty of mathematics.
“McKinsey Structured Thinking” — any problem should be structurally decomposed. When I encounter new problems, I think this way. A useful book.
“The Chrysanthemum and the Sword” — stock from years ago that I dug out to read. An American’s post-WWII perspective on Japan. You can glimpse aspects of Japanese culture like modified Confucianism without “benevolence (ren),” the psychology of indebtedness, etc. One drawback is it’s quite dated — modern Japan is largely different from that era.
“The Black Swan” — a black swan refers to unforeseen extreme events. Black swan events will always happen — there’s no such thing as 100% accurate prediction. It also discusses classification, which reminded me of content from “Structured Thinking” and “The Worlds I See”: “The essence of human understanding is classifying things,” but classification always awkwardly leaves some things unclassifiable or unable to be classified. Black swan events exist from the moment of classification. An interesting and noteworthy reflection.
“The Professional” — by Kenichi Ohmae. Very mediocre, not recommended.

Spiritual / Self-Help Book List:

“The Evolution of Desire” — evolutionary psychology, a masterpiece.
“Die with Zero” — experience the right things at different life stages. Even if you revisit something after missing it, it won’t feel the same as experiencing it at the right time. A life manual, highly recommended.
“Ten Minutes Meditation” — mainly about the importance of meditation and how to do it. I learned meditation through this book. When I first completed meditation, I fell in love with it. It gave me a feeling of being taken to outer space and then returning to Earth. More importantly, it truly relieves stress. Meditation has become part of my life.
“The Manipulation Bible” — okay.
“Siddhartha” — incomprehensible, rubbish.
“The Book of Life” — pure chicken soup, rubbish.

Fiction Book List:

“The Stranger” — a masterpiece. An indescribable sense of authenticity, feeling like an outsider oneself.
“Yellowface” — a very interesting book about a white American woman who plagiarizes an unpublished work by a deceased Asian writer, even using a very Chinese pen name. When fans discover she’s white, you can feel the embarrassment. Playfully explores racial prejudice. As thrilling as watching a TV drama — twists and turns, gripping. Highly recommended.
“The World of Yesterday” — by Stefan Zweig. Austria, Europe, WWI and WWII through a writer’s eyes. Returning to that turbulent Europe from a different angle. A very good book.
“Project Hail Mary” — sci-fi. I increasingly dislike reading sci-fi. This one is okay: imagine you’re on an alien exploration mission, all your crewmates have died, and you happen to encounter a friendly alien. How do you communicate with them…
“Letter from an Unknown Woman” — by Stefan Zweig. Not good. Only the first story is somewhat novel. No interest in seriously reading the other two.
“Satantango” — incomprehensible. Even Nobel Prize in Literature winners vary in quality.

Blog and WeChat Official Account
#

The name of my WeChat Official Account has always been a struggle. I didn’t put much thought into maintaining it anyway, so I casually used a few names. This year I watched a documentary — “The Last Porter” (最后的棒棒), which moved me deeply. The DBA profession, like the porters of Chongqing, is undergoing tremendous change. So I simply changed it to “最后的DBA” (The Last DBA). This name rolls off the tongue nicely and carries some historical context and philosophical reflection. Seems like a good name.

Since a lot of time goes into work, I didn’t have much time for writing to begin with. Plus, this year my operational approach kept changing, and no matter how I adjusted my daily schedule, I couldn’t carve out a good time slot. I even invested some money, and my time still didn’t increase, which frustrated me for quite a while. Looking back now, I only published 12 articles this year — not even one in the first half of the year.

Very dissatisfied. 😠

I don’t know if my skills have improved or if the system is genuinely stable, but cases worth deep research seem to have become fewer. But this isn’t really a big problem. This year I also started treating paper interpretation as an article type. I personally feel the results are decent — I can learn quite a bit, without being too insular or reinventing the wheel. Using AI to interpret papers would certainly be fast, but I personally feel there are two problems:

Do I truly understand? I feel like I don’t — it’s not the same concept as reading through it myself. Reading it yourself not only allows deeper understanding but also lets you discover all sorts of quirky details.
Can’t pad articles. If I can interpret a paper with one prompt, then I feel the dissemination value is minimal — surely there’s no one not using AI now, right?

Of course, I don’t read every paper word by word — that would be too inefficient. I only select papers that I feel are good and worth frame-by-frame interpretation, and savor them carefully.

A quick summary of this year’s articles:

Too few in quantity
Slightly improved quality, and useful content (several articles I’m personally very satisfied with)
Explored new formats

Final Thoughts
#

This year was a busy one, with both bad and good memories. Many important things were left unfinished. Next year should bring significant changes. Writing this year-end summary is quite interesting — looking back to see what my past selves were up to is a fun experience.

Last year’s 2025 OKRs:

Continue some things — FAILED
Think about how to produce output — FAILED
Master another track — HALF SUCCESSFUL
PostgreSQL… haven’t figured out what more to do — FAILED
Find a way to resume fitness — FAILED

2026 Plan:

Continue some things
Pay attention to my psychological and physical health — next year’s annual health inspection alerts should be lower than this year’s
Pay attention to article readership, maintain the WeChat Official Account
Explore DB AI Ops, report to myself next year
Manage upward — don’t invest too much time in work
Travel during holidays instead of grinding
Read no fewer than 30 books, but don’t focus solely on quantity

My 2024 Year-End Summary

Sat, 11 Jan 2025 00:00:00 +0000

As a DBA
#

2023 was a year of comprehensive PostgreSQL learning for me, and 2024 has been a year of comprehensive PostgreSQL operations. There’s actually a lot of material I really want to dive into but haven’t had the time. This year was mainly case analysis — I could only supplement my foundational knowledge here and there.

Mid-year there was a discussion about “will DBAs be eliminated in the cloud era.” This discussion left a deep impression on me. I thought about many things afterward — why do others seem to have so few things to deal with while I, as a DBA, have so much? I even went into cloud computing groups to debate about it, and I actually gained something from it. Different perspectives lead to unexpected conclusions. The conclusion of the debate may boil down to just one thing: DBAs are providing 1510 emotional value to their leaders.

Right or wrong, you can see reflections on the DBA profession in many of my articles this year. Continuing down the traditional DBA path is certainly a dead end. Today’s DBAs lean more toward business data layer operations, or moving up to architecture design. Positions for expert DBAs focused purely on databases are actually very few.

READING
#

Let me reiterate why I’m so devoted to reading (I said this in 2023 too…):

The value brought by reading is immeasurable in the short term
Reading brings a pleasant sense of intellectual enrichment
Learning is a belief. Yuval Harari has a view: believing in science is actually also a form of faith. I choose to believe in this faith, at least in 2024 and the foreseeable future.

My book list roughly falls into three categories: PostgreSQL, broader technical scope, and extracurricular. Some are in Chinese, some in English. Some are physical books, some electronic.

This year, let me continue with a reading list ranking. Horizontal comparison across different categories is a bit of a stretch, so let’s compare within categories. Once again, note: these book lists are for books I aimed to “finish cover to cover.” Books used as references don’t count here.

2024 PostgreSQL Book List (ranked by preference):

“PostgreSQL Database Kernel Analysis” — clear thinking and framework, though the version is a bit old
“Quickly Mastering PostgreSQL Version New Features” — this should be my favorite PostgreSQL book this year, because it has zero fluff throughout, a pleasure to read
“The Internals of PostgreSQL” — I originally wanted to put this first, but since there’s a free online version at interdb, I wouldn’t even recommend buying this book. It’s ranked here because interdb is so excellent — its substitute is enshrined here as a deity
“The Way of PostgreSQL: From Apprentice to Expert, 2nd Edition” — very detailed but also very long. I recommend skimming through quickly to find the key points without lingering too long
“PostgreSQL Technical Internals: Transaction Processing Deep Dive” — transactions are the foundation of PostgreSQL, and also the foundation of my source code journey
“PostgreSQL in Action” — the practical examples are well worth referencing
“PostgreSQL 16 Administration Cookbook” — not recommended. The table of contents framework looks good, but the content is hollow. Don’t waste time on this book.

2024 Broader Technical Scope Book List (ranked by preference):

“DDIA-v2: Designing Data-Intensive Applications (2nd Edition)” — so good I don’t know where to begin. So excellent that I specially wrote reading notes (my only book notes article this year). I wish I had found it sooner.
“A Brief History of Databases” — reading history truly brings insight. The story of databases begins here. Some technical things become clearer in hindsight.
“ITIL 4 and DevOps Service Management Certification Guide (2nd Edition)” — a classic in IT service management. It elevated my understanding of the operations role — how did these things so closely tied to my work come about? Which parts don’t match reality, and why weren’t they applied? You can grasp many things from it.
“Cloud Native Kubernetes” — hardcore, another track entirely
“Docker Deep Dive” — decent for understanding containers and container history. The container knowledge itself isn’t actually that much.
“Brother Bird’s Linux Private Kitchen” — sorry, I genuinely hadn’t read this classic. Came to catch up. The writing approach is well worth learning from. The drawback is that much of it isn’t useful for my role.
“Machine Learning” — ranked here not because the book is bad, but because it’s very hard to understand. I gave up about a quarter of the way through. This book showed me the upper limits of my intelligence, and I’m sad about it.
“Building a Vector Database from Scratch” — if you want to read source code, go to GitHub
“Deep Understanding of Go Language” — understood nothing at all

2024 Extracurricular Book List (ranked by preference):

“Cancer Ward” — I finished this early in the first half of the year. While reading it, I felt: barring surprises, this book would rank first this year. Nobel Prize in Literature, well-deserved.
“Intimate Relationships” — understanding relationships with lovers, friends, and bosses. Academic paper style, solid, I like it.
“Does God Play Dice? A History of Quantum Physics” — setting aside everything else, the writing style provides immense emotional value, making me want to keep reading. I finished it in just a few days.
“The Worlds I See” — AI pioneer Fei-Fei Li’s autobiography. The story of a girl who grew up in Chengdu venturing into the melting pot of America, eventually leading Google AI, while also narrating the history of AI development.
“21 Lessons for the 21st Century” — the final installment of Yuval Harari’s trilogy. I loved the first two books, but this one felt just okay. At least it brought closure.
“The Old Man and the Sea” — hard to evaluate. I like its temperament, but not its content.
“The Wandering Earth” — this is a collection of Liu Cixin’s short stories. One day at the library, I bought it because of the first short story. After buying it, I found the other short stories to be very boring and childish. I felt cheated.
“Journey to the West” — hot take: they can’t even explain Tang Sanzang’s background properly. A mess, completely confused. I gave up after a little bit. (My evaluation of “Romance of the Three Kingdoms” last year was very high.)

Blog and WeChat Official Account
#

2024 Published Articles:

PostgreSQL technical: 21
Other technical: 2
Book notes: 1
Useless articles: 1

I only wrote 25 articles this year, a noticeable decrease from last year.

WeChat Official Account followers: 600. Though not many, I believe every single one has good taste 😸

Writing technical articles is actually quite tiring — it takes far more time than one would imagine. However, you genuinely learn things during the writing process, and the sense of accomplishment from completing a piece is real. Since I feel responsible for my articles, I won’t write recklessly about things I don’t understand. As for errors arising from misunderstandings, that’s actually normal. No one can guarantee that their future self won’t criticize their current self — just write correctly for the current state.

In terms of writing content this year, I gave up writing reading notes for extracurricular books. I wrote quite a few last year, but writing reading notes takes a lot of time with very low value. Low emotional value tasks naturally get abandoned. In fact, my writing content varies each year. Currently, PostgreSQL database technical articles are the only constant — other types aren’t as stable. This is normal. The blog was originally meant for database writing. If there’s no application scenario for other domains, I won’t touch them again after the brief exploratory period.

One more complaint: domestic blogging platforms only care about article quantity, which is completely at odds with my writing style. Each of my articles is tens of thousands of hand-typed characters. I’m a quality-over-quantity blogger. So I can’t be bothered anymore — I’m planning to abandon CSDN in 2025 and just post on GitHub and my WeChat Official Account.

I’ve been writing on CSDN since 2017. When I first started blogging, there weren’t many good blog hosting platforms. Looking at CSDN now: community interaction is zero, and the vast majority of articles on it are terrible. Even I don’t want to find CSDN articles myself. It’s like a first love of 7-8 years — sometimes you just have to break up.

2024 Publication Channels:

CSDN Blog: https://liuzhilong.blog.csdn.net
Modb.pro: liuzhilong62
GitHub: https://github.com/liuzhilong62/blogs
WeChat Official Account: 破斯特贵斯库儿

Expected 2025 Channels:

GitHub: https://github.com/liuzhilong62/blogs
WeChat Official Account: 破斯特贵斯库儿
Other platforms: we’ll see

Final Thoughts
#

I seem to talk about work-learning balance every year… Due to a dramatic increase in workload this year, there was even a period where I couldn’t study at all. Balance has been shattered. Not having time to study is unacceptable to me, so I later adjusted my daily schedule (thanks to “Atomic Habits” — I absolutely love this book), and finally managed to squeeze out some study time. Actually, as long as no one’s around, learning efficiency is high.

I’ve collected some quotes I resonated with this year:

Don’t let others become dependencies in your task chain –heisenberg.liu
Plans that require execution are generally simple plans –heisenberg.liu
Things not implemented equal things not done –somebody
Solve problems yourself instead of waiting for others to reply –somebody
Important things should be done immediately — waiting even a moment means they won’t get done –somebody
Don’t do repetitive low-value tasks. Think more about the context behind this requirement –heisenberg.liu
Don’t pan for gold in shit. Find ways to get quality information sources –somebody
SREs need the ability to configure optimal default parameters and the ability to modify these parameters in bulk –“Enterprise Cloud Computing”
The more miscellaneous tasks you do, the more miscellaneous tasks come your way –heisenberg.liu
SREs spend 50% of time on operations and 50% on development –“Enterprise Cloud Computing”
Premature optimization is the root of all evil. Premature code abstraction is also the root of all evil –somebody
The speed at which the human brain receives knowledge is limited –somebody
If someone won’t let you read, leave that person or leave that environment –heisenberg.liu
Teams that build knowledge bases are slackers –somebody
The value of a standard is determined by the customer –“ITIL 4”
Heroism: working long hours and troubleshooting alone. Long working hours also lead to burnout with the work itself. Those who want to be heroes are only interested in their own achievements and turn a deaf ear to team collaboration –“ITIL 4”
Not all problems need root cause analysis. It depends on the frequency of occurrence and the scope of the failure –“ITIL 4”

Looking back at the plans I set for myself in 2023: only 2 items total, and I completed neither. KPI achievement rate: 0% 😄

Combining agile operations, agile project management, and OKR thinking: setting a full-year plan for myself at the beginning of the year is simply unreasonable. Looking back at last year and the year before, some of my plans emerged mid-way and won priority battles over other tasks. And some tasks simply couldn’t be completed — this should be a normal state. So, I won’t set too many flags for myself.

2025 Plan:

Continue some things
Think about how to produce output
Master another track
PostgreSQL… haven’t figured out what more to do
Find a way to resume fitness

Getting Started with pg_rewind

Tue, 13 Aug 2024 00:00:00 +0000

What is pg_rewind?
#

pg_rewind is a PostgreSQL-provided tool. When the timelines of two PG instances diverge, pg_rewind can synchronize them. (For example, the primary is running, the standby failover has been running for a while — at this point the primary and standby timelines have diverged.)

pg_rewind compares the sizes of files between the source and target, then copies differing files from source to target, including configuration files. However, it does not compare unchanged files, so pg_rewind runs efficiently on large databases with few changes.

pg_rewind can be used after a standby failover: even if the standby has been running independently for some time, it can be pulled back to the same state as the primary and become a standby again.

During execution, pg_rewind compares the divergence point between primary (source) and standby (target), and transmits the primary’s WAL logs after the divergence point to the standby. Therefore, if the primary’s WAL after the divergence point is also lost, rewind won’t copy nonexistent WAL logs, and the standby will still fail to become a standby. The solution is to use restore.

!!! When using pg_rewind, back up the target instance. pg_rewind directly overwrites the target database’s files. If rewind fails, the target database may be unable to start.

Using pg_rewind
#

After a primary-standby switchover, the old primary continues running, causing timeline inconsistency. The old primary cannot start as a standby for the new primary.

When attempting to start the standby, a timeline error appears:

LOG: entering standby mode
FATAL: requested timeline 2 is not a child of this server's history
DETAIL: Latest checkpoint is at 0/6000028 on timeline 1, but in the history of the requested timeline, the server forked off from that timeline at 0/4000098.
LOG: startup process (PID 22321) exited with exit code 1
LOG: aborting startup due to startup process failure
LOG: database system is shut down

At this point, rewind is needed to realign the primary and standby.

Configure pg_hba on the current primary Set up login permissions for the pg_rewind user to access the source database. hba changes require a database restart.

vi $source/pg_hba.conf
host all pg 172.17.100.150/32 trust

pg_rewind requires a high-privilege user. Newer PG versions allow granting privileges; older versions should use a superuser. My environment is PG 9.6, so I use the OS superuser directly.

wal_log_hints = on parameter configuration Append wal_log_hints = on to the target database’s postgres.conf, then start and shut down the target database once (at this point the primary is running and the standby is shut down).

vi $dest/postgres.conf

wal_log_hints = on

Execute pg_rewind

[pg@lzl pg96data_sla]$ /pg/pg96/bin/pg_rewind --target-pgdata /pg/pg96data_pri --source-server='host=172.17.100.150 port=5433 user=pg password=oracle dbname=postgres'
servers diverged at WAL position 0/4000098 on timeline 1
rewinding from last common checkpoint at 0/4000028 on timeline 1
Done!

Configure standby parameters Modify IP, port, directory, etc. in postgres.conf and recovery.conf. pg_rewind also copies configuration files over.

[pg@lzl pg96data_pri]$ mv recovery.done recovery.conf
[pg@lzl pg96data_pri]$ vi recovery.conf
[pg@lzl pg96data_pri]$ vi postgres.conf

Start the standby

[pg@lzl pg96data_pri]$ /pg/pg96/bin/pg_ctl -D /pg/pg96data_sla -l /pg/pg96data_sla/server.log start 
server starting
[pg@lzl pg96data_sla]$ psql -p5433 postgres
psql (9.6.17)
postgres=# \x
Expanded display is on.
postgres=# select * from pg_stat_replication ;
-[ RECORD 1 ]----+------------------------------
pid | 24766
usesysid | 16384
usename | lzl
application_name | walreceiver
client_addr | 172.17.100.150
client_hostname | 
client_port | 47345
backend_start | 2021-07-30 07:44:05.582546+00
backend_xmin | 
state | streaming
sent_location | 0/4033790
write_location | 0/4033790
flush_location | 0/4033790
replay_location | 0/4033790
sync_priority | 0
sync_state | async

Common Issues
#

pg_rewind Error 1
#

could not fetch remote file "global/pg_control": ERROR: must be superuser to read files
Failure, exiting

Solution: Use a high-privilege user.

postgres=# \du
 List of roles
 Role name | Attributes | Member of 
-------------+------------------------------------------------------------+-----------
 lzl | Replication | {}
 pg | Superuser, Create role, Create DB, Replication, Bypass RLS | {}
 rewind_user | | {}

The pg user is the built-in superuser that comes with the PG server, matching the PG installation user. The OS installation user certainly has permission to modify pg_control.

/pg/pg96/bin/pg_rewind --target-pgdata /pg/pg96data_pri --source-server='host=172.17.100.150 port=5433 user=pg password=oracle dbname=postgres'

pg_rewind Error 2
#

could not connect to server: FATAL: no pg_hba.conf entry for host "172.17.100.150", user "rewind_user", database "postgres"
Failure, exiting

No pg_hba.conf entry configured for the connection. Solution: Configure pg_hba for the user, e.g.:

host all pg 172.17.100.150/32 trust

pg_rewind Error 3
#

[pg@lzl pg96data_sla]$ /pg/pg96/bin/pg_rewind --target-pgdata /pg/pg96data_pri --source-server='host=172.17.100.150 port=5433 user=pg password=oracle dbname=postgres'

target server needs to use either data checksums or "wal_log_hints = on"

Root causes:

full_page_writes (enabled by default)
wal_log_hints must be set to on, or PG must have checksums enabled at initdb time.

Solution: Add wal_log_hints = on to the target database’s postgres.conf, then start and shut down the target database once (the target was already shut down — it must be started and shut down again for the parameter to take effect).

vi postgres.conf # add to target database config

wal_log_hints = on

Restart the target database to apply:

[pg@lzl pg96data_sla]$ /pg/pg96/bin/pg_ctl -D /pg/pg96data_pri -l /pg/pg96data_pri/server.log start 
server starting
[pg@lzl pg96data_sla]$ /pg/pg96/bin/pg_ctl -D /pg/pg96data_pri -l /pg/pg96data_pri/server.log stop
waiting for server to shut down.... done

References
#

https://www.postgresql.org/docs/9.6/app-pgrewind.html

How I Got Into Wuhan University's Part-Time Master's Program

Tue, 13 Aug 2024 00:00:00 +0000

Why Did I Want to Pursue a Part-Time Master’s?
#

To improve my academic credentials. My undergraduate degree is from an ordinary university. A higher degree can add a bit of competitiveness in my career.
I once submitted my resume to a state-owned enterprise and was completely ghosted. But a colleague with better academic credentials in the same office got through. So for state-owned enterprises, higher education is the knock on the door.
To make up for failing the graduate entrance exam as a senior and revive the dream of graduate studies.
Learning is never wrong — this is my creed.

Differences Between Full-Time and Part-Time Graduate Programs
#

Study Mode
#

Full-time means you quit your job; part-time allows you to keep working. This basically locks in part-time as the only option for most working people.

Exam Scope
#

Full-time exams are more demanding, usually covering four subjects: advanced math, graduate English, politics, and a specialized course. Part-time exams are less demanding, covering two subjects: the Management Comprehensive Exam (middle school math, logic, writing) and graduate English. Except for English, which is similar to the full-time version, the management comprehensive exam content is much easier than the full-time track (more on this later).

Research Direction
#

Full-time graduate programs lean toward research, emphasizing learning and research output, cultivating students’ learning and research abilities.

Part-time programs lean toward enhancing students’ management skills, delivering management-oriented talent to society.

These two directions are quite different.

Social Recognition
#

Full-time graduate degrees certainly carry more recognition than part-time ones. After all, the bar is higher, the study pressure is greater, it’s the mainstream path, and society recognizes it more. But part-time degrees do carry recognition too — many schools have explicitly stated they treat both equally (on paper). Most importantly, part-time graduate students hold dual certificates (degree certificate and diploma).

As for employment, it depends on the employer. Some positions only require any graduate degree, while others may explicitly state “full-time graduate degree required.” But for those who can’t quit their jobs to pursue higher education, part-time is practically the only path. In summary: Part-time also grants dual certificates, but part-time recognition < full-time recognition.

How to Choose a Major
#

Consider your job nature, career aspirations, and your wallet.

HR, finance, or corporate executive: MBA

Technical roles or engineering management: MEM

Civil servant or public administration: MPA

Accounting: MPAcc. There are a few other niche options — search online.

From a financial perspective, tuition varies by school but generally follows similar ranges. Taking Sichuan University as an example: MEM costs about 15,000 yuan per year, MBA about 150,000 yuan per year, MPA roughly similar to MEM.

For an IT professional like me, with a thin wallet and not seeing myself as an executive, MEM is the better fit.

How to Choose a School
#

Since the study difficulty is relatively low and social recognition isn’t as high as full-time, I recommend choosing a prestigious local university. 211 and 985 universities are strongly recommended — just pick one you like. Many 985 universities set their admission cutoff at the national line, so I personally feel that non-211/985 schools aren’t worth applying to. If the scores are the same, why not choose a better school?

Of course, some 985 universities set their own cutoff lines. You’ll need to check the school’s department website for historical admission scores. For example, Sichuan University sets its own line every year, typically 20–30 points above the national line.

How Does the Exam Work?
#

Exam Content
#

The exam is divided into the preliminary exam and the re-examination. The preliminary exam is in late December; the re-examination is in March.

The preliminary exam is a written test. After registering for the exam and selecting a test venue, you take it in late December — finished in one day, each session 3 hours.

The re-examination is an interview. A few schools add a written component, but since the pandemic, it’s all been online interviews — rarely do you need to write anything during the interview.

Preliminary exam content (Management Comprehensive Exam):

Re-examination content:

Early Interview
#

The early interview means the school arranges an interview before the preliminary exam — effectively moving the re-examination earlier. Once you pass the early interview, you only need to reach the national line on the preliminary exam. Under the normal process, you’d need to exceed the school’s own cutoff line.

Early interviews are only offered by some schools. For example, Tsinghua has an early-admission interview; Sichuan University doesn’t. You’ll need to check the official website of your target school.

If you pass the early interview, the pressure on the preliminary exam is indeed lighter.

How to Register
#

The two most important websites in the graduate exam process are your target school’s official website and the China Graduate Admission Website (研招网, YZW).

Before registering, check the master’s program catalog for your target school and major. For example, part-time engineering management should be selected as follows:

Should You Sign Up for a Training Course?
#

Many people wonder whether to sign up for a training course. Signing up feels too expensive — what if you don’t pass? Not signing up means you don’t know how to study, or studying feels too exhausting.

I have some authority on this question, because I did sign up for one.

I saw a training course online and asked about the price — 8,000 yuan. On top of that, there was an information gap: I didn’t know what the exam covered, how to study, how to register, which school to apply to, or where to search for this information. (Searching “part-time graduate” on Baidu immediately yields nothing but ads.) Plus, I was genuinely determined to study at the time. So I fell into this trap…

What Did the Training Course Give Me?
#

First, a pile of study materials — study methods, past exam papers, and so on. Other than the English vocabulary list, which I immediately started memorizing, I barely touched anything else. I printed the past exam papers too, but I never looked at them — not even by the time the exam was over. They looked useful, but in reality, you can search past papers on Taobao and find plenty of officially published versions with detailed explanations — far more useful and less straining on the eyes. And that vocabulary list — strangely, the one the course gave me didn’t match the one in Zhang Jian’s Yellow Book. I memorized the course’s vocabulary for a long time, only to find that some common exam words weren’t in the list. Later I switched to Zhang Jian’s Yellow Book vocabulary and it felt much better.

Beyond study materials, the most important component was live-streamed lectures, typically 8–10 PM — two hours of teaching and ten minutes of Q&A.

The live lectures were useful, especially logic and math. Just listening to those two subjects essentially eliminated the need to buy extra books to thoroughly study the fundamentals of math and logic — you only needed to do the post-class exercises and practice problems. I barely listened to the English classes; I mostly self-studied. Personally, I felt that listening to English lectures was very inefficient and a waste of time — better to memorize more words and do more reading exercises. I only listened to the last two sessions of English writing, which were extremely useful. More on English writing later (with practical tips). Finally, don’t fantasize about asking the teacher questions — these online classes have many students, and the Q&A time is only about ten minutes. My questions were never picked.

Benefits of a training course: Convenience of learning. From a working person’s perspective, you’re already working overtime a lot. Coming home exhausted, expecting yourself to spread out materials and study like it’s the gaokao — too hard. But if it’s a lecture, you just sit on the sofa and watch the livestream. That’s much easier. Saves time. No need to laboriously make a study plan and constantly adjust it. Listening to lectures is also easier than reading through a thick textbook on your own. Essentially, a training course is trading money for time.

The good learning state of classmates motivates you. You’re not studying alone with no idea how others are doing.

The pitfalls of training courses: Quality varies widely. The institution I signed up with was Shangde. I didn’t research them beforehand — they were quite mediocre. Some of their programs are contract-based: they refund if you don’t pass, but there were traps in the contract, and no one got refunds. Our group had many people fighting for refunds. Also, personal information leaks — basically every student received refund scam calls. Even I, who passed, got five or six such calls.

Teacher quality is uneven. Some teachers were excellent; others seemed like they were just coasting. Some explanations were outright misleading. In my class, the math and logic teachers were especially good, English was garbage, and writing was misleading…

Don’t fantasize that a training course will train you into a great candidate. The course is only an aid — it mainly depends on you. From the day I started planning for the exam until the preliminary exam was over, I had basically zero weekends — every one was spent in the library or a café. I declined every social gathering.

So, should you sign up for a training course?

I think if you meet all of the following conditions, you can consider it:

Enough resolve. Since you’ve paid, don’t let it go to waste. I also don’t recommend refund-based programs that give you an escape route.
Enough money. Online courses start at a few thousand yuan — my 8,000 yuan can serve as a reference. In-person courses are more expensive but offer face-to-face tutoring.
Unable to bridge the information gap. The information gap may prevent you from planning your own study schedule. If the information gap is what drives you toward a course, I suggest looking at others’ study plans and successful Bilibili uploaders’ cases. The biggest source of information is always the school’s official website.

If you have the time, energy, insufficient funds, or decent learning ability, you absolutely don’t need to sign up. In that case, making a study plan that suits you is especially important.

The Preliminary Exam
#

Before the preliminary exam is over, focus only on preparing for the preliminary exam. Generally speaking, you can prepare for the re-examination content after the preliminary exam is done. Preparing for the preliminary exam is the core of your studies, the most energy-consuming and competitive phase — this is where success is decided.

How to Prepare for the Preliminary Exam — Study Plan
#

To prepare for the preliminary exam, you need a study plan that fits you, and you need to throw yourself into it completely.

The study plan is extremely, extremely, extremely important. You need to first examine yourself — what are your strengths and weaknesses, your circumstances, which subjects you’re unfamiliar with, and which ones need long-term study.

Everyone’s situation is different. Let me first share my study plan — you can reference my approach to customizing a plan and my study methods. Because the pressure isn’t as high (compared to full-time), I strongly recommend starting in July or August. Starting too late means not enough time; starting too early makes it easy to slack off. Total study time should be 5–6 months. But if your English is really poor, start memorizing words a few months earlier.

My personal conditions:

Not enough time — often worked overtime until 9 PM, weekends generally off. Commute by subway, two hours total both ways.
Math almost completely forgotten, never touched logic before, Chinese writing has been terrible since childhood, decent English vocabulary, reading comprehension fine, English writing completely unable.

Given the study pressure and my personal conditions, my plan needed to be:

Memorize words. English is definitely the most time-consuming — it requires sustained, long-term vocabulary memorization. Before all other studying, memorize English II vocabulary. Since morning memory retention is best, I memorized words on the subway to work every day and also on weekend mornings. From August until the preliminary exam.
English reading. Actually, once you’ve memorized the words, reading comprehension is easy. English II doesn’t have many long, complex sentences — if you know all the words, reading is no problem. But I personally enjoy English reading, so I scheduled daily reading of English originals. I can’t say it had a huge impact, but it wasn’t useless — consider it a supplement to exam prep. Importantly, hobbies make habits easier to form.
Math and logic have similar study difficulty. Even though I was completely clueless, they were relatively easy to learn (the concepts are simple; the exam itself is another story — but more on that later). I studied math or logic from 8 PM to 10 PM on weekday evenings (mostly attending lectures — if you don’t have lectures, buy materials and self-study). This is also long-term study: early phase learning concepts, late phase practicing. Since I couldn’t always leave work on time, sometimes I had to use the evening commute and the one-hour lunch break to complete the daily math and logic tasks. (Never fall behind — one day of delay leads to a huge backlog.)
Chinese writing. Prepare about one month before the exam — late November or early December. Look at writing materials and try writing yourself. Don’t aim for perfection — the main thing is to express the core idea clearly. Trust me, during the exam you’ll absolutely be writing in frantic cursive.
English writing. Prepare about one month before the exam. Remember: absolutely, absolutely do not memorize model essays. Not only is it brutally hard to memorize them, they’re nearly impossible to adapt. Memorize 2–3 templates before the exam, then practice with past exam topics using the templates. You only need to swap in words — no situation where you pick up the pen and can’t write a single word.

So, my weekly plan:

This arrangement felt quite suitable for me — it fully utilized fragmented time and made good use of weekends. The first 3–4 months build the foundation: English’s foundation is vocabulary, logic and math’s foundation is concepts. Weekly study on workdays could be consolidated and practiced on weekends. The final 1–1.5 months are mainly for writing and getting the feel of past papers.

Recommended Study Materials
#

English: Zhang Jian’s Yellow Book. Just buy the vocabulary book and past exam papers. Baicizhan (vocabulary app), iReading, WeChat public account: 考研英语外刊 (Graduate English Foreign Journals).
English writing: I don’t recommend any purchasable writing guide. Use universal templates; don’t memorize model essays.
Math: Chen Jian’s Math High Score Guide, past exam answer keys.
Logic: Zhonggong’s Logic Easy Pass, past exam answer keys.
Management comprehensive writing: Buy a popular one — they’re all not great. No need to master writing too deeply.

Don’t buy practice problem books — buy past papers directly. The quality of existing practice problems doesn’t compare at all to past papers. For English, no need to buy practice books — directly buy past papers. For math and logic, beyond the built-in exercises that come with foundational study, don’t buy extra practice problem books. I did math practice problems for a short period — very time-consuming and ineffective. The key for math and logic is to solidify the fundamentals, cover all the concepts, then do past papers and review the answer explanations. In short, immersive study time (like weekends) should only be spent on past papers. Do the last 20 years’ worth of papers, then cycle through them again. (20 sets of past papers — only 2 per weekend — takes over two months to finish one round; by the time you redo them, you’ve largely forgotten the earlier ones.) Always save the most recent two years’ papers untouched — use them for timed self-testing two weeks before the exam.

English Study
#

Vocabulary
#

Morning vocabulary memorization, rain or shine (it works especially well on the subway…). Baicizhan — some people like using it. I used it early on too, but I found it ineffective. It covers the entire vocabulary pool; after a year you may not even complete one full cycle, and you’ve long forgotten what you studied earlier. So I stopped using it later. I strongly recommend my personal vocabulary method.

Everyone’s vocabulary is different. At the start, you must go through all the words once (graduate exam vocabulary is about 5,000 words) and pull out the ones you don’t know onto a vocabulary list. Since carrying a vocabulary notebook on the subway is slightly awkward, I put it on my phone.

I have a dedicated vocabulary photo album:

When memorizing, open it, zoom in — effectively covering the definitions while memorizing.

Cycle through like this. At first I did 2 pages a day, advancing 1 page a day. Later, 4 pages a day, advancing 4 pages a day. No matter what, cycle through — memorize until you can cover the definition and know the word’s meaning. For words easily confused, add them to the list, take another photo, and update the album.

Before my preliminary exam, I had cycled through these words six or seven times. Basically, aside from beyond-syllabus words, there was nothing I didn’t know.

Reading
#

English total score: 100. Reading ability components = Cloze 10 points + Reading Comprehension 40 points + New Question Type Reading 10 points = 60 points. No matter how poor your English ability, reading comprehension cannot be weak. My reading ability mostly came from daily foreign journal reading, such as iReading and 考研英语外刊. About 20 minutes a day, light study pressure. (Actually, it’s mainly vocabulary — if you know the words, sentences are easy to understand.)

iReading: “Love the World” foreign journals, one passage a day. Under Reading Plan → More Collections → Foreign Journals → Love the World, subscribe to the monthly issues, one a day. Relatively easy, good for early-stage reading improvement.
WeChat public account: 考研英语外刊, one passage a day. Updated daily. This account is very well done, highly recommended. Just harder — good for later-stage challenge. If you don’t fully understand, that’s fine; I sometimes couldn’t fully grasp it either, since the difficulty is a bit high.

Read along with morning vocabulary memorization. If short on time, you can also read on the way home.

Long and complex sentences: English II doesn’t have many. Some people dedicate time specifically to studying them. If you want to study long sentences specifically, I especially recommend Liu Xiaoyan’s Long Sentences video series (just search on video sites — they’re all free). It’s very engaging and well-organized, easy to stick with. As for me, I only watched the simple sentence part of Liu Xiaoyan’s course and stopped. Because, first, I found that as long as you know the words, you basically understand the sentences; second, the videos are too long and numerous, taking up too much study time.

Writing
#

Writing is divided into short composition (letter or notice, 10 points) and long composition (data analysis essay — bar chart/pie chart analysis, 15 points).

Again, do NOT memorize model essays. Before the exam I bought a writing book and memorized 10 model essays — truly, truly excruciating to memorize, and impossible to adapt. After memorizing the model essays, the first time I attempted an English writing question, I couldn’t write a single word — no exaggeration.

The most valuable thing in my training course was the English templates. Using the templates, I worked through all the past years’ English writing topics — every single one could be adapted. The number of words to swap in doesn’t exceed twenty; you just need to be able to write simple sentences. Here are the templates:

Short composition template — Letter:

Dear Sir or Madam,

 I am an undergraduate who majos in Applied English in this/a university.I am writing this letter for the purpose of doing sth.

 1.It,first an formost,is my idea that not only ... but also

 2.Then more importantly,so ... that...

 3.The last on I must point out is that 简单句,which could be accepted by the majority of 人/.

 So It is the very moment for me to do ...,And I am looking forward to your reply.

 yours truly,
 xxx.

Where “doing sth” includes:

感谢信:expressing my genuine gratitude for your kind help
建议信:making some suggestions concerning sth.
投诉信:making my complaints concerning sth.
祝贺信:show my sincere congratulations to you because 句子
道歉信:offer my sincere apology to you because 句子
邀请信:invite you to participate in 活动 on behalf of 某人/组织
通知信:have 某人 informed that 句子

The letter template works for all types of letters.

Besides letters, the short composition may — with low probability — test notices. The notice format differs from letters.

Short composition template — Notice:

 Notice

 In an effort to do sth,I woud like to offer you some detailed information about it.

 The 活动 will be held in the school auditorium at 7 p.m.,next Saturday,December 28th and the requirements for sth. are listed as follows.

 主段内容同书信...

 If you have any questions,please feel free to send on email to
studentsunion@123.com or call 1234567.We are looking forward to your participation.

The long composition essentially only involves analyzing bar charts and pie charts. The data falls into two categories: comparing magnitudes and comparing trends. Only the first paragraph differs between the two; the latter two paragraphs are the same.

Long composition template:

 (比大小首段)The diagram clearly shows/illustrates/d that 句子/词组(the purposes of/attitudes toward/the proportions of) among participants/respondents in a certain college.
Based on the data offered,one can distinctly see that 对象1 ranks the first/highest among all the categories,accounting for 数据1.Next are 对象2 and 对象3 with 数据2 and
数据3 respectively ,while 对象4 only constitutes 数据4.
 (比趋势首段)The diagram clearly illustrates how 话题 changed during the past several years.Based on the data provided,one can distinctly see that the number of 对象1 rose/fell significantly/slightly/gradually from 数据 in 年 to 数据 in 年,while that the number of 对象2 experienced a gradual/significant increase/decrease during the same period,reaching 数据 in 年.
 From my standpoint,there are two fundamental factors that are responsible for this scence.To begin with,the first contributing factor is that 句子.In addition,another important factor that cannot be ignored is that 句子.
 In view of the analysis above,we can conclude that it is of little surprise to see this phenomenon in the current era.Therefore,it can be predicted that 名词词组/动词ing will still take up a large share in the future.

Writing without templates — relying on your own ability — is extremely difficult. Getting an ultra-high score with templates is hard, but getting 70–80% of the score is no problem, and the upfront investment is basically zero. After memorizing the templates, just write through all the past years’ writing topics once.

Math and Logic Study
#

Math questions:
#

Logic questions:
#

Math and logic are both multiple choice — nothing special to say. Early phase: learn concepts. Later phase: improve speed.

Math and logic have a huge number of concepts. The early study phase takes 3–4 months, 2–3 hours a day, to learn all the concepts. After mastering the concepts, practice with past papers and review answer explanations. In the final month, practice with a stopwatch to improve speed: math questions within 70 minutes, logic within 60 minutes.

Chinese Writing Study
#

The management comprehensive essay is divided into Argument Validity Analysis and Argumentative Essay.

Argument Validity Analysis is essentially nitpicking — get a writing guide and look through it; it’s not hard. You need to find the logical flaws in a lengthy passage of material. When writing, find four problem points. If you haven’t studied it, you might struggle to find them; after studying, finding four points is fairly easy. Don’t worry about naming the flaws precisely — overgeneralization, equivocation, false dichotomy, etc. Just write “xxx does not lead to xxx.”

The Argumentative Essay mainly involves interpreting a short passage. The key is not to misinterpret the theme. Finding the theme is also challenging at first — look at more sample materials to get a feel for it; generally you can locate the theme. The standard structure is introduction-body-conclusion. I recommend using “individual – enterprise – nation” as the framework (intro – individual – enterprise – nation – conclusion, 5 paragraphs total). Pick a few tried-and-tested points to plug in. Some students with strong writing skills write argumentative essays using other approaches — I certainly admire that. But writing time is extremely limited. Unless you’re naturally gifted with lightning-fast thinking, I recommend using a formulaic approach. Finishing the essay is the top priority.

Exam Time Strategy
#

Yes, you need to strategize the exam timing too. Trust me 100% — you will not finish the management comprehensive exam. It’s the most time-crunched exam I’ve ever taken. You know you can solve the problems, but you have no time to compute.

On exam day: morning — management comprehensive, 3 hours. Afternoon — English, 3 hours.

English: 3 hours, relatively little content, no need for repeated recalculation — time is completely sufficient. When I finished, I had 50 minutes left and left early.

Management comprehensive: 3 hours, absolutely not enough. In my pre-exam self-timed simulations, I consistently took 4 hours. During the real exam, for math — any question over 3 minutes, skip immediately. If it feels computationally heavy, skip immediately. For logic — absolutely cannot use your usual analytical approach. Speed-read the question (logic questions have colossal amounts of text), look at the options, pick whatever feels right. Logic questions requiring computation: temporarily abandon, come back later if time permits. For writing — read the prompt and start writing immediately. Write as fast as you possibly can (both essays combined no more than 1 hour). Every 2 minutes saved could rescue a multiple-choice question.

You don’t have to finish all the multiple-choice (do fill in the answer sheet completely though), but you MUST finish the writing. So the question order is important. Many people do the essays first, then multiple-choice. I did math first, then essays, then logic. Either way, don’t leave writing for the end. In the real exam, both essays must be finished within 55 minutes — 1,500 words total, plus reading the prompt and brainstorming. Try it once and you’ll know how impossibly short the time is.

The last 20 minutes: fill in the answer sheet. After filling it in, continue solving problems.

The Re-examination
#

Basic Information About the Re-examination
#

If you’ve made it past the national line or the school’s own cutoff — congratulations, you’ve completed 90% of the journey. The remaining 10% is the re-examination. The re-examination has a mandatory elimination rate (required by national policy), typically around 70–80% passing rate. Since elimination must exist, some people get cut every year. If you don’t prepare, you’re very likely to be among them. Here’s a joke: I got cut from Sichuan University’s re-examination~

Re-examination timeframe: mid-to-late March each year.

Score release: mid-March.

Content: spoken English, specialized knowledge, comprehensive interview, politics (Sichuan University: open-book politics, no need to prepare. Wuhan University: closed-book written politics…)

Since the pandemic, re-examinations have been online interviews — no written test environment. Experts ask questions; you answer.

So you have three months to prepare for the re-examination. Conveniently, the preliminary exam ends late December, scores aren’t out yet, and February is Chinese New Year — realistically, most people start preparing only when scores are released. Take me as a cautionary example: I received the re-examination notice on March 21, the re-examination was on March 27 — I had six days to prepare, including spoken English and engineering management, which I’d never touched before… So it was embarrassing: I couldn’t answer a single one of the examiner’s English questions, couldn’t answer a single specialized question. Cut from the re-examination.

Post-Adjustment (Tiaoji)
#

When I found out I’d failed Sichuan University’s re-examination, my mood plummeted. But — every cloud has a silver lining. The adjustment (tiaoji) process was my lifesaver. While searching for adjustment schools, I found Wuhan University.

The China Graduate Admission Website has a dedicated adjustment window, giving students who failed their initial re-examination three more interview opportunities. You can fill in three preferences — three schools to apply to. Since each school has different re-examination dates and requirements, preparing for all of them is very hard. I focused mainly on preparing for Wuhan University’s adjustment. The adjustment, of course, also involves a re-examination — essentially, schools that haven’t filled their enrollment quotas run the process again, giving students who weren’t admitted in the first round another chance.

Adjustment window: late March to early April.

How to Prepare for the Re-examination?
#

The re-examination is also highly competitive. Lazy people like me are not uncommon… But no matter what, you’ve already invested over half a year — you can’t let it go down the drain. (I almost did…) For non-specialist students like me, the hardest parts of the re-examination are spoken English and specialized knowledge. From score release to the re-examination, you have about one week (while still working!), so learning from scratch is impossible. Based on my experience, the following approaches, in descending order of importance:

Find seniors who’ve been through it and get past re-examination materials (discreetly — sharing re-examination materials externally is prohibited) and course materials. See if anyone you know is at that school, or find groups on forums or Tieba.
Search Bilibili for common graduate re-examination questions. Summarize them and memorize.
Buy the school’s recommended reference books (usually course materials). They’re thick; you won’t finish them.

Finally, the most important thing: mock re-examination. Summarize potential English questions, specialized questions, and comprehensive interview questions, then find a partner to act as the examiner for a mock interview.

There are other re-examination requirements — keep an eye on department updates and your email: score weightings, interview process, dual-camera setup, interview schedule, document preparation, etc.

The End
#

The 2022 national preliminary exam line was 185. My preliminary score was 210 (English 80, Management Comprehensive 130). Here’s my re-examination acceptance notice ^_^

Good luck to all working-student-warriors battered by society but still holding onto your dreams — may your graduate exam go smoothly. You’ve got this!!!

OGG Oracle-to-PostgreSQL Sync — Hands-On Steps

Tue, 13 Aug 2024 00:00:00 +0000

Source DB: Oracle (11.2.0.4) 192.168.10.141 Target DB: PGSQL (10.12) 192.168.10.128 OGG software version: (19.1.0.0.4) OGG download: Oracle GoldenGate Downloads glibc issue handling: https://www.cnblogs.com/hxlasky/p/16779047.html

1. Install OGG Software on Source and Target
#

Source:

A. Configure response file: oggcore.rsp

oracle.install.responseFileVersion=/home/oracle/oggcore.rsp
INSTALL_OPTION=ORA11g
SOFTWARE_LOCATION=/oracle/ogg
START_MANAGER=false
MANAGER_PORT=7809
DATABASE_LOCATION=/oracle/db/11.2.0.4
INVENTORY_LOCATION=/oracle/oraInventory
UNIX_GROUP_NAME=oinstall

B. Silent install OGG

./runInstaller -silent -nowait -responseFile /home/oracle/oggcore.rsp

oracle@szgtsp431-or@ecsdb>./runInstaller -silent -nowait -responseFile /home/oracle/oggcore.rsp
Starting Oracle Universal Installer...
Checking Temp space: must be greater than 120 MB. Actual 32405 MB Passed
Checking swap space: must be greater than 150 MB. Actual 2048 MB Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2020-08-14_08-57-27AM. Please wait ...
You can find the log of this install session at:
 /oracle/oraInventory/logs/installActions2020-08-14_08-57-27AM.log
Successfully Setup Software.
The installation of Oracle GoldenGate Core was successful.
Please check '/oracle/oraInventory/logs/silentInstall2020-08-14_08-57-27AM.log' for more details.

2. Set Database to Archive Mode
#

oracle@szgtsp431-or@ecsdb>sqlplus / as sysdba
SQL*Plus: Release 11.2.0.4.0 Production on Fri Aug 14 09:06:34 2020
Copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
SQL> archive log list;
Database log mode Archive Mode
Automatic archival Enabled
Archive destination /oracle/oradata/archivelog
Oldest online log sequence 19
Next log sequence to archive 21
Current log sequence 21

3. Enable Force Logging and Minimum Supplemental Logging
#

alter database force logging;
alter database add supplemental log data;
alter system switch logfile;

Verify force logging and minimum supplemental logging enabled:

select force_logging,supplemental_log_data_min from v$database;

4. Set enable_goldengate_replication Parameter
#

alter system set enable_goldengate_replication=true scope=both;

If RAC, all nodes must be modified:

alter system set enable_goldengate_replication=true scope=both sid='*';

5. Create OGG User, Tablespace, and Grant Privileges
#

create tablespace tbs_ogg datafile '/oracle/oradata/datafile/tbs_ogg01.dbf' size 100M;
create user goldengate identified by 123456 default tablespace tbs_ogg temporary tablespace temp;
grant create session,alter session to goldengate;
grant alter system to goldengate;
grant resource to goldengate;
grant connect to goldengate;
grant select any dictionary to goldengate;
grant flashback any table to goldengate;
grant select any table to goldengate;
grant select any table to goldengate;
grant insert any table to goldengate;
grant update any table to goldengate;
grant delete any table to goldengate;
grant select on dba_clusters to goldengate;
grant execute on dbms_flashback to goldengate;
grant create table to goldengate;
grant create sequence to goldengate;
grant alter any table to goldengate;
grant dba to goldengate;
grant lock any table to goldengate;

6. Enable Table-Level Supplemental Logging
#

To sync table data from specific schemas, enable supplemental logging on those tables.

Check supplemental logging:

SELECT owner, table_name, log_group_name, log_group_type,
 decode(always, 'ALWAYS', 'Unconditional', NULL, 'Conditional') always
FROM dba_log_groups
ORDER BY owner, table_name, log_group_name;

Enable supplemental logging during low-activity window:

oracle@szgtsp431-or@ecsdb>ggsci
Oracle GoldenGate Command Interpreter for Oracle
Version 19.1.0.0.4 OGGCORE_19.1.0.0.0_PLATFORMS_191017.1054_FBO
Linux, x64, 64bit (optimized), Oracle 11g on Oct 17 2019 23:13:12
Operating system character set identified as US-ASCII.
Copyright (C) 1995, 2019, Oracle and/or its affiliates. All rights reserved.
GGSCI (szgtsp431-or) 1> dblogin userid goldengate,password 123456
Successfully logged into database.
GGSCI (szgtsp431-or as goldengate@ecsdb) 2> add trandata ecs.*
2020-08-14 09:13:54 INFO OGG-15132 Logging of supplemental redo data enabled for table ECS.DEPT.
2020-08-14 09:13:54 INFO OGG-15133 TRANDATA for scheduling columns has been added on table ECS.DEPT.
2020-08-14 09:13:54 INFO OGG-15135 TRANDATA for instantiation CSN has been added on table ECS.DEPT.
2020-08-14 09:13:54 INFO OGG-15132 Logging of supplemental redo data enabled for table ECS.INFO.
...

Verify all supplemental logging added:

select * from (
 select owner,table_name from dba_tables where owner in ('BGLWT')
 minus
 select owner,table_name from dba_log_groups)
order by owner,table_name;
-- no rows selected = all table-level supplemental logging added successfully

7. Configure Manager Process
#

oracle@szgtsp431-or@ecsdb>ggsci
...
GGSCI (szgtsp431-or) 1> dblogin userid goldengate,password 123456
Successfully logged into database.
GGSCI (szgtsp431-or as goldengate@ecsdb) 2> create subdirs
Creating subdirectories under current directory /home/oracle
...
GGSCI (szgtsp431-or as goldengate@ecsdb) 3> edit param mgr
PORT 7809
DYNAMICPORTLIST 7810-7980
PURGEOLDEXTRACTS ./dirdat/*, USECHECKPOINTS, MINKEEPDAYS 3
PURGEDDLHISTORY MINKEEPDAYS 7, MAXKEEPDAYS 10
LAGREPORTHOURS 1
LAGINFOMINUTES 30
LAGCRITICALMINUTES 45

8. Configure Extract Process
#

GGSCI (szgtsp431-or as goldengate@ecsdb) 7> add extract extecs, tranlog, threads 1,begin now
EXTRACT added.
GGSCI (szgtsp431-or as goldengate@ecsdb) 8> add exttrail ./dirdat/lt, extract extecs
EXTTRAIL added.
GGSCI (szgtsp431-or as goldengate@ecsdb) 9> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING 
EXTRACT STOPPED EXTECS 00:00:00 00:00:38 
GGSCI (szgtsp431-or as goldengate@ecsdb) 10> edit param extecs
EXTRACT extecs
SETENV (ORACLE_HOME = "/oracle/db/11.2.0.4")
SETENV (ORACLE_SID = "ecsdb")
USERID goldengate, PASSWORD 123456
EXTTRAIL ./dirdat/lt
TRANLOGOPTIONS EXCLUDEUSER goldengate
TRANLOGOPTIONS DBLOGREADER
DBOPTIONS ALLOWUNUSEDCOLUMN
FETCHOPTIONS USESNAPSHOT, USELATESTVERSION, MISSINGROW REPORT
STATOPTIONS REPORTFETCH
WARNLONGTRANS 1h, CHECKINTERVAL 10m
DYNAMICRESOLUTION
DISCARDFILE ./dirrpt/extecs.dsc, APPEND, MEGABYTES 1024
DISCARDROLLOVER AT 6:00
REPORTROLLOVER AT 6:00
REPORTCOUNT EVERY 1 MINUTES, RATE
DDL INCLUDE MAPPED
DDLOPTIONS ADDTRANDATA, REPORT
DDLOPTIONS NOCROSSRENAME, REPORT
TABLE ECS.*;

9. Configure Pump Process
#

GGSCI (szgtsp431-or as goldengate@ecsdb) 11> add extract deliecs, exttrailsource ./dirdat/lt
EXTRACT added.
GGSCI (szgtsp431-or as goldengate@ecsdb) 12> add rmttrail ./dirdat/rt, extract deliecs, megabytes 500
RMTTRAIL added.
GGSCI (szgtsp431-or as goldengate@ecsdb) 13> edit param deliecs
EXTRACT deliecs
PASSTHRU
DYNAMICRESOLUTION
RMTHOST 192.168.10.100, MGRPORT 7809
RMTTRAIL ./dirdat/rt
DISCARDFILE ./dirrpt/deliecs.dsc, APPEND, MEGABYTES 1024
DISCARDROLLOVER AT 6:00
REPORTCOUNT EVERY 1 MINUTES, RATE
REPORT AT 0:00
REPORT AT 1:00
...
REPORT AT 23:00
REPORTROLLOVER AT 00:00
STATOPTIONS RESETREPORTSTATS
TABLE ECS.*; 

10. Start Extract Process
#

GGSCI (szgtsp431-or as goldengate@ecsdb) 20> start extecs
Sending START request to MANAGER ...
EXTRACT EXTECS starting
GGSCI (szgtsp431-or as goldengate@ecsdb) 21> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING 
EXTRACT STOPPED DELIECS 00:00:00 00:06:06 
EXTRACT RUNNING EXTECS 00:00:00 00:00:01 

11. Configure Target OGG Software
#

A. Upload OGG software and extract B. Configure OGG environment variables

[pgsql@szgtsp428-or ~]$ vi .bash_profile
## .bash_profile
## Get the aliases and functions
if [ -f ~/.bashrc ]; then
 . ~/.bashrc
fi
## User specific environment and startup programs
PATH=$PATH:$HOME/bin
export PATH
export PGHOME=/usr/local/pgsql
export PGDATA=/data/pgsql
export OGG_HOME=/data/ogg
export PATH=$PATH:$PGHOME/bin:$OGG_HOME
LD_LIBRARY_PATH=$PGHOME/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib:$OGG_HOME/lib
export LD_LIBRARY_PATH
export ODBCINI=/home/pgsql/odbc.ini
export DD_ODBC_HOME=/data/ogg

[pgsql@szgtsp428-or ~]$ ggsci
Oracle GoldenGate Command Interpreter for PostgreSQL
Version 19.1.0.0.200714 OGGCORE_19.1.0.0.0OGGBP_PLATFORMS_200628.2141
Linux, x64, 64bit (optimized), PostgreSQL on Jun 29 2020 03:59:15
Operating system character set identified as UTF-8.
Copyright (C) 1995, 2019, Oracle and/or its affiliates. All rights reserved.
GGSCI (szgtsp428-or) 1>

12. Create Database and Table on Target
#

ecsdb=# \l
 List of databases
 Name | Owner | Encoding | Collate | Ctype | Access privileges 
-----------+----------+----------+-------------+-------------+-------------------
 ecsdb | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | 
 postgres | pgsql | UTF8 | en_US.UTF-8 | en_US.UTF-8 | 
 template0 | pgsql | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/pgsql +
 | | | | | pgsql=CTc/pgsql
 template1 | pgsql | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/pgsql +
 | | | | | pgsql=CTc/pgsql
(4 rows)

ecsdb=# \d
 List of relations
 Schema | Name | Type | Owner 
--------+--------------+-------+----------
 public | student_info | table | postgres
(1 row)

ecsdb=# select * from student_info;
 id | name | address 
----+------+---------
 1 | Zhang San | Guangzhou
 2 | Li Si | Shenzhen
 3 | Wang Wu | Shanghai
 4 | Zhao Liu | Beijing
 5 | Sun Qi | Wuhan
 6 | A Da | Chengdu
 7 | A Er | Nanjing
(7 rows)

13. Configure Target Manager Process and Start
#

[pgsql@szgtsp428-or ogg]$ ggsci
Oracle GoldenGate Command Interpreter for PostgreSQL
...
GGSCI (szgtsp428-or) 1> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER STOPPED 
GGSCI (szgtsp428-or) 2> create subdirs
Creating subdirectories under current directory /data/ogg
...
GGSCI (szgtsp428-or) 3> edit param mgr
port 7809
GGSCI (szgtsp428-or) 4> info all
...
GGSCI (szgtsp428-or) 5> start mgr
Manager started.
GGSCI (szgtsp428-or) 7> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING 

Now start the pump process on source (deliecs):

oracle@szgtsp431-or@ecsdb>ggsci
...
GGSCI (szgtsp431-or) 1> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING 
EXTRACT ABENDED DELIECS 00:00:00 01:06:41 
EXTRACT RUNNING EXTECS 00:00:00 00:00:07 
GGSCI (szgtsp431-or) 2> start deliecs
Sending START request to MANAGER ...
EXTRACT DELIECS starting
GGSCI (szgtsp431-or) 3> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING 
EXTRACT RUNNING DELIECS 00:00:00 01:06:55 
EXTRACT RUNNING EXTECS 00:00:00 00:00:01 

14. Target PostgreSQL Parameter Adjustment
#

wal_level = logical #minimal, replica, or logical
max_replication_slots = 10 #max number of replication slots
max_wal_sender = 10 #maximum number of wal sender processes
wal_receiver_status_interval=10s #optional, keep the system default
wal_sender_timeout = 60s #optional, keep the system default
track_commit_timestamp=off #optional, keep the system default

Restart PostgreSQL after adjusting parameters:

[pgsql@szgtsp428-or pgsql]$ pg_ctl stop -D /data/pgsql/ -l /data/pgsql/logfile
waiting for server to shut down.... done
server stopped
[pgsql@szgtsp428-or pgsql]$ pg_ctl start -D /data/pgsql/
waiting for server to start.... done
server started

15. Data Source Configuration (odbc.ini)
#

[ODBC Data Sources]
PGDSN=DataDirect 10.12 PostgreSQL Wire Protocol
postgres=DataDirect 10.12 PostgreSQL Wire Protocol
scott=DataDirect 10.12 PostgreSQL Wire Protocol
[ODBC]
IANAAppCodePage=4
InstallDir=/data/ogg
[PGDSN]
Driver=/data/ogg/lib/GGpsql25.so
Description=DataDirect 10.12 PostgreSQL Wire Protocol
Database=ecsdb
HostName=127.0.0.1
PortNumber=5432
LogonID=postgres
Password=123456

16. Connection Test
#

[pgsql@szgtsp428-or ~]$ cd /data/ogg
[pgsql@szgtsp428-or ogg]$ ggsci
...
GGSCI (szgtsp428-or) 1> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING 
GGSCI (szgtsp428-or) 2> dblogin sourcedb pgdsn userid postgres, password postgres
2020-08-14 11:35:01 INFO OGG-03036 Database character set identified as UTF-8. Locale: en_US.UTF-8.
2020-08-14 11:35:01 INFO OGG-03037 Session character set identified as UTF-8.
Successfully logged into database.

17. Configure and Start Replicat Process on Target
#

Add checkpoint table:

GGSCI (szgtsp428-or) 1> dblogin sourcedb pgdsn userid postgres, password 123456
Successfully logged into database.
GGSCI (szgtsp428-or as postgres@pgdsn) 2> add checkpointtable public.chkt
Successfully created checkpoint table public.chkt.

Configure replicat:

GGSCI (szgtsp428-or as postgres@pgdsn) 34> edit param repl
REPLICAT repl
SOURCEDEFS ./dirdef/student_info.def
SETENV (PGCLIENTENCODING = "UTF8")
SETENV (ODBCINI="/home/pgsql/odbc.ini")
SETENV (NLS_LANG="AMERICAN_AMERICA.AL32UTF8")
targetdb pgdsn userid postgres, password 123456
DISCARDFILE ./dirrpt/repl.dsc, purge
MAP ecs.student_info, TARGET public.student_info;

GGSCI (szgtsp428-or as postgres@pgdsn) 36> add replicat repl,exttrail ./dirdat/rt,checkpointtable public.chkt
REPLICAT added.
GGSCI (szgtsp428-or as postgres@pgdsn) 38> start repl
Sending START request to MANAGER ...
REPLICAT REPL starting

GGSCI (szgtsp428-or as postgres@pgdsn) 55> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING 
REPLICAT RUNNING REPL 00:00:00 00:00:08

18. Test Verification
#

First, create matching table structure on target:

create table student_info (id int primary key, name varchar(100), address varchar(100));

Then initialize data:

Configure extinit process on source:

GGSCI (szgtsp431-or as goldengate@ecsdb) 17> edit param extinit
EXTRACT extinit
userid goldengate, PASSWORD 123456
REPORTCOUNT EVERY 30 MINUTES, RATE
DISCARDFILE ./dirrpt/extinit.dsc, APPEND, MEGABYTES 1024
RMTHOST 192.168.10.100,MGRPORT 7809, compress
RMTTASK replicat,GROUP replinit
TABLE ecs.student_info;

GGSCI (szgtsp431-or as goldengate@ecsdb) 18> ADD EXTRACT extinit, SOURCEISTABLE
EXTRACT added.

Configure replinit process on target:

GGSCI (szgtsp428-or as postgres@pgdsn) 28> edit param replinit
REPLICAT replinit
targetDB pgdsn, USERID postgres, PASSWORD 123456
discardfile ./dirrpt/replinit.dsc, PURGE
SOURCEDEFS ./dirdef/student_info.def
Map ecs.student_info,target public.student_info;

GGSCI (szgtsp428-or as postgres@pgdsn) 29> add replicat repinit, SPECIALRUN
REPLICAT added.

Start Oracle-to-PG data initialization:

GGSCI (szgtsp431-or as goldengate@ecsdb) 9> start extinit
Sending START request to MANAGER ...
EXTRACT EXTINIT starting

Target: (view initialization row count via View report replicat)

Check both sides:

Source (Oracle):

SQL> select * from student_info;
 ID NAME ADDRESS
---------- ---------- ----------
Zhang San Guangzhou
Li Si Shenzhen
Wang Wu Shanghai
Zhao Liu Beijing
Sun Qi Wuhan
A Da Chengdu
A Er Nanjing
A San Beijing
rows selected.

Target (PostgreSQL):

ecsdb=# select * from student_info;
 id | name | address 
----+------+---------
 1 | Zhang San | Guangzhou
 2 | Li Si | Shenzhen
 3 | Wang Wu | Shanghai
 4 | Zhao Liu | Beijing
 5 | Sun Qi | Wuhan
 6 | A Da | Chengdu
 7 | A Er | Nanjing
 8 | A San | Beijing
(8 rows)

Insert data on source:

SQL> insert into ecs.student_info values (10,'aa','bb');
1 row created.
SQL> commit;
Commit complete.

Check target — data synchronized successfully.

Original link: https://lastdba.com/2024/08/13/ogg搭建oracle-pg同步实操步骤/

OGG PostgreSQL-to-Oracle Sync — Hands-On Steps

Tue, 13 Aug 2024 00:00:00 +0000

OGG software version: (19.1.0.0.4) Oracle version: 11.2.0.4 PG version: pg10 OGG download: https://www.oracle.com/technetwork/middleware/goldengate/downloads/index.html

glibc issue handling: https://www.cnblogs.com/hxlasky/p/16779047.html

1. Create Database and Table on Source
#

[root@node2 ~]# su - postgres
Last login: Tue Jul 21 21:08:52 CST 2020 on pts/0
[postgres@node2 ~]$ pg_ctl -D /opt/pgsql_data -l logfile start
waiting for server to start.... done
server started
postgres=# create database test
postgres=# \c lzldb
postgres=# create table tab1(id int primary key,name varchar(20))

2. Create Database and Table on Target
#

sqlplus / as sysdba
SQL> create table ORALZL.tab1(id number primary key,name varchar2(20));

3. Extract and Install OGG for PostgreSQL
#

-- Unlike OGG for Oracle, OGG for PG only needs extraction. Oracle version requires running runInstaller.
[postgres@node1 ~]$ id postgres
uid=54323(postgres) gid=54330(postgres) groups=54330(postgres)
[postgres@node1 ~]$ exit
logout
[root@node1 ~]# mkdir /ogg
[root@node1 ~]# chown -R postgres /ogg
[root@node1 ~]# chmod -R 755 /ogg
[root@node1 ~]#
[root@node1 soft]# ls -l

total 240744

-rw-r--r--. 1 root root 87028695 Jul 22 02:51 19100200714_ggs_Linux_x64_PostgreSQL_64bit.zip
[root@node1 soft]# chmod 777 19100200714_ggs_Linux_x64_PostgreSQL_64bit.zip
[root@node1 soft]# unzip 19100200714_ggs_Linux_x64_PostgreSQL_64bit.zip
Archive: 19100200714_ggs_Linux_x64_PostgreSQL_64bit.zip
 inflating: ggs_Linux_x64_PostgreSQL_64bit.tar 
 inflating: OGG-19.1.0.0-README.txt 
 inflating: release-notes-oracle-goldengate_19.1.0.200714.pdf 
[root@node1 soft]# chmod 777 ggs_Linux_x64_PostgreSQL_64bit.tar
[root@node1 soft]# su - postgres
[postgres@node1 ~]$ cd /soft
[postgres@node1 soft]$ tar -xf ggs_Linux_x64_PostgreSQL_64bit.tar -C /ogg

4. Configure PG User Environment Variables
#

Source PG:

[postgres@node1 ~]$ cat .bash_profile
## .bash_profile
## Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
## User specific environment and startup programs
PATH=$PATH:$HOME/.local/bin:$HOME/bin
export GGHOME=/ogg
export PG_DATA=/opt/pgsql/pgsql/bin
export PATH=$PG_DATA:$PATH
export PG_HOME=/opt/pgsql/pgsql
export LD_LIBRARY_PATH=$PG_HOME/lib:$LD_LIBRARY_PATH:$GGHOME/lib
export ODBCINI=/home/postgres/odbc.ini
export DD_ODBC_HOME=/ogg
export PATH
[postgres@node1 ~]$ source .bash_profile

5. Configure Manager Process
#

[postgres@node1 ~]$ cd /ogg
[postgres@node1 ogg]$ ./ggsci
Oracle GoldenGate Command Interpreter for PostgreSQL
Version 19.1.0.0.200714 OGGCORE_19.1.0.0.0OGGBP_PLATFORMS_200628.2141
Linux, x64, 64bit (optimized), PostgreSQL on Jun 29 2020 03:59:15
Operating system character set identified as UTF-8.
Copyright (C) 1995, 2019, Oracle and/or its affiliates. All rights reserved.
GGSCI (node1) 2> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER STOPPED 

GGSCI (node1) 3> create subdirs
Creating subdirectories under current directory /ogg
Parameter file /ogg/dirprm: created.
Report file /ogg/dirrpt: created.
Checkpoint file /ogg/dirchk: created.
Process status files /ogg/dirpcs: created.
SQL script files /ogg/dirsql: created.
Database definitions files /ogg/dirdef: created.
Extract data files /ogg/dirdat: created.
Temporary files /ogg/dirtmp: created.
Credential store files /ogg/dircrd: created.
Masterkey wallet files /ogg/dirwlt: created.
Dump files /ogg/dirdmp: created.
GGSCI (node1) 4> edit params mgr
GGSCI (node1) 5> view params mgr
port 7809
GGSCI (node1) 6> start mgr
Manager started.
GGSCI (node1) 7> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING 

6. Adjust Source PostgreSQL Parameters
#

[postgres@node1 ogg]$ vi /opt/pgsql_data/postgresql.conf
wal_level = logical #minimal, replica, or logical
max_replication_slots = 10 #max number of replication slots
max_wal_sender = 10 #maximum number of wal sender processes
wal_receiver_status_interval=10s #optional, keep the system default
wal_sender_timeout #optional, keep the system default
track_commit_timestamp #optional, keep the system default
wal_receiver_status_interval=10s
wal_sender_timeout = 60s
track_commit_timestamp=off

Restart source PostgreSQL after adjustment:

[postgres@node1 ogg]$ pg_ctl -D /opt/pgsql_data -l logfile stop
[postgres@node1 ogg]$ pg_ctl -D /opt/pgsql_data -l logfile start

7. Configure OGG for PG Data Source
#

cd /home/postgres/
vi odbc.ini
[ODBC Data Sources]
PGDSN=DataDirect 7.1 PostgreSQL Wire Protocol
postgres=DataDirect 7.1 PostgreSQL Wire Protocol
scott=DataDirect 7.1 PostgreSQL Wire Protocol
[ODBC]
IANAAppCodePage=4
InstallDir=/ogg
[PGDSN]
Driver=/ogg/lib/GGpsql25.so
Description=DataDirect 7.1 PostgreSQL Wire Protocol
Database=test
HostName=192.168.1.112
PortNumber=5432
LogonID=postgres
Password=postgres

8. Connection Test
#

[postgres@node1 ~]$ cd /ogg
[postgres@node1 ogg]$ ./ggsci
--dblogin sourcedb pgdsn userid pg, password 123456
GGSCI (node1) 1> dblogin sourcedb pgdsn userid postgres, password postgres
2020-07-22 03:10:44 INFO OGG-03036 Database character set identified as UTF-8. Locale: en_US.UTF-8.
2020-07-22 03:10:44 INFO OGG-03037 Session character set identified as UTF-8.
Successfully logged into database.
GGSCI (node1 as postgres@pgdsn) 2>

9. Enable Table-Level Supplemental Logging
#

Source:

GGSCI (node1) 3> dblogin sourcedb pgdsn userid postgres, password postgres
2020-07-22 03:21:01 INFO OGG-03036 Database character set identified as UTF-8. Locale: en_US.UTF-8.
2020-07-22 03:21:01 INFO OGG-03037 Session character set identified as UTF-8.
Successfully logged into database.
GGSCI (node1 as postgres@pgdsn) 4> add trandata public.tab1 --If table has primary key, this step can be skipped
Logging of supplemental log data is enabled for table public.tab1. REPLICA IDENTITY was DEFAULT and is changed to FULL
GGSCI (node1 as postgres@pgdsn) 5>
GGSCI (node1 as postgres@pgdsn) 5> info trandata public.tab1
Logging of supplemental log data is enabled for table public.t1 with REPLICA IDENTITY set to FULL

10. Register Extract Process on PG
#

Registering an extract process on PG essentially creates a replication slot. The output plugin defaults to test_decoding.

 GGSCI (node1 as postgres@pgdsn) 6> Register Extract ext_pg
 2020-07-22 03:25:27 INFO OGG-25355 Successfully created replication slot 'ext_pg_2947c06e0ea2ec74' for EXTRACT group 'EXT_PG' in database 'test'.

11. Configure Extract and Pump Processes
#

Configure extract process:

 edit param ext_pg

SETENV ( PGCLIENTENCODING = "UTF8" )
 SETENV (NLS_LANG="AMERICAN_AMERICA.AL32UTF8")
 extract ext_pg
 SETENV (ODBCINI="/home/pg/odbc.ini" ) 
 SOURCEDB pgdsn, USERID pg, PASSWORD 123456
 exttrail ./dirdat/st
 TABLE PUBLIC.TAB1;
 ----GETTRUNCATES ### This feature on PostgreSQL 10.12: ERROR OGG-25541 GETTRUNCATES is not valid. PostgreSQL supports TRUNCATE capture from version 11.

Note: PG to Oracle cannot sync TRUNCATE commands.

Configure pump process:

 extract pump_pg
 SETENV (ODBCINI="/home/pg/odbc.ini" )
 RMTHOST 172.17.100.150, MGRPORT 7809, compress
 numfiles 10000
 RMTTRAIL ./dirdat/rt
 TABLE PUBLIC.TAB1;

12. Add Trail and Start Extract/Pump
#

ADD extract ext_pg, TRANLOG,BEGIN now
 add exttrail ./dirdat/st,extract ext_pg,megabytes 500
 add extract pump_pg,exttrailsource ./dirdat/st
 add rmttrail ./dirdat/rt,extract pump_pg,megabytes 500
 start ext_pg
 start pump_pg

13. Configure defgen
#

If table structures are consistent, you can configure ASSUMETARGETDEFS.

edit param defgen

DEFSFILE ./dirdef/tab1.def, PURGE
 SOURCEDB pgdsn, USERID pg, PASSWORD 123456
 TABLE PUBLIC.tab1;

Generate table definition file:

defgen paramfile /oggpg/dirdef/tab1.prm

Copy the defgen file to the target’s dirdef directory.

14. Verify Trail Delivery on Target
#

[oracle@lzl dirdat]$ cd dirdat
 [oracle@lzl dirdat]$ ll
 -rw-r----- 1 pg pg 1439 Feb 28 11:02 rt000000000

15. Register Extract Process on PG
#

Registering an extract process on PG creates a replication slot:

GGSCI (node1 as postgres@pgdsn) 6> Register Extract ext_pg
2020-07-22 03:25:27 INFO OGG-25355 Successfully created replication slot 'ext_pg_2947c06e0ea2ec74' for EXTRACT group 'EXT_PG' in database 'test'.

16. Configure Oracle User Environment Variables
#

export ORACLE_BASE=/oracle/app/oracle
 export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/dbhome_1
 export ORACLE_SID=oralzl
 export OGG_HOME=/oggfororacle
 export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:$PATH
 export TNS_ADMIN=$ORACLE_HOME/network/admin
 export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$OGG_HOME:$ORACLE_HOME/lib32:/lib/usr/lib:/usr/local/lib

17. Configure Oracle Listener and TNS
#

OGG for Oracle defaults to using TNS_ADMIN’s tns. You can also manually configure during extract configuration, e.g.: USERID goldengate@127.0.0.1:1521/oralzl, PASSWORD 123456

18. Install OGG for Oracle on Target
#

Download OGG software. Configure oggcore.rsp file:

oracle.install.responseFileVersion=/home/oracle/oggcore.rsp
INSTALL_OPTION=ORA11g
SOFTWARE_LOCATION=/ogg
START_MANAGER=false
MANAGER_PORT=7809
DATABASE_LOCATION=/oracle/db/11.2.0.4
INVENTORY_LOCATION=/oracle/oraInventory
UNIX_GROUP_NAME=oinstall

Silent install OGG:

./runInstaller -silent -nowait -responseFile /home/oracle/oggcore.rsp

19. Oracle Database User and Privileges
#

create user goldengate identified by "123456";
 grant create session,alter session to goldengate;
 grant alter system to goldengate;
 grant resource to goldengate;
 grant connect to goldengate;
 grant select any dictionary to goldengate;
 grant flashback any table to goldengate;
 grant select any table to goldengate;
 grant select any table to goldengate;
 grant insert any table to goldengate;
 grant update any table to goldengate;
 grant delete any table to goldengate;
 grant select on dba_clusters to goldengate;
 grant execute on dbms_flashback to goldengate;
 grant create table to goldengate;
 grant create sequence to goldengate;
 grant alter any table to goldengate;
 grant dba to goldengate;
 grant lock any table to goldengate;

20. Target Manager Process
#

edit param mgr

PORT 7809
 DYNAMICPORTLIST 7810-7980
 PURGEOLDEXTRACTS ./dirdat/*, USECHECKPOINTS, MINKEEPDAYS 3
 PURGEDDLHISTORY MINKEEPDAYS 7, MAXKEEPDAYS 10
 LAGREPORTHOURS 1
 LAGINFOMINUTES 30
 LAGCRITICALMINUTES 45

start mgr

21. Configure Replicat Process on Target
#

GGSCI (node2) 8> dblogin userid goldengate@127.0.0.1:1521/oralzl,password 123456
GGSCI (node2 as postgres@pgdsn) 9> add checkpointtable goldengate.chkt
Successfully created checkpoint table public.chkt.

Replicat process:

 edit param rep_pg
REPLICAT rep_pg
 USERID goldengate@127.0.0.1:1521/oralzl, PASSWORD 123456
 SOURCEDEFS ./dirdef/tab1.def
 MAP public.tab1, TARGET oralzl.tab1; 
add replicat rep_pg,exttrail ./dirdat/rt,checkpointtable goldengate.chkt
 start rep_pg

22. Test Sync
#

[postgres@node1 ~]$ psql
postgres=# \c lzldb
test=# \d tab1;
​ Table "public.tab1"
 Column | Type | Collation | Nullable | Default 
--------+-----------------------+-----------+----------+---------
 id | integer | | not null | 
 name | character varying(20) | | | 
Indexes:
 "t1_pkey" PRIMARY KEY, btree (id)


lzldb=# insert into t2 values(1,'lzl1') ;
INSERT 0 1
lzldb=# select * from t2;
 id | name 
----+------
 1 | lzl1


[postgres@node2 ~]$sqlplus / as sysdba
SQL> select * from oralzl.tab1; 
​ id name 
---------- ----------
​ 1 lzl1 

Original link: https://lastdba.com/2024/08/13/ogg搭建pg-oracle同步实操步骤/

Getting Started with HikariCP Connection Pool

Mon, 12 Aug 2024 00:00:00 +0000

A Brief Introduction to HikariCP
#

“Hikari” means “light” in Japanese — HikariCP aims to be a Connection Pool as light and fast as light. This nearly Java-only middleware connection pool is extremely lightweight and performance-focused. HikariCP is now the default connection pool for Spring Boot, and with the proliferation of Spring Boot and microservices, HikariCP usage continues to grow.

On the HikariCP GitHub homepage, there’s a performance comparison: （https://github.com/brettwooldridge/HikariCP-benchmark）

It appears to crush all other database connection pool middleware. However, this performance comparison is somewhat dated and lacks a comparison with Alibaba’s homegrown pinnacle connection pool, Druid. I briefly checked Druid’s GitHub page — it actually has slightly more stars than HikariCP. Druid is clearly stronger in terms of functionality. As for which has better performance, it even sparked a spat between experts, and I haven’t seen any rigorous performance comparison report yet. But that’s not the focus of this article… this article is just to get a basic understanding of HikariCP.

Key Connection Pool Parameters
#

There aren’t that many parameters. Let’s pick the important ones:

Parameter	Meaning
minimumIdle	This property controls the minimum number of idle connections HikariCP tries to maintain in the pool. If the number of idle connections drops below this value and the total number of connections in the pool is less than maximumPoolSize, HikariCP will do its best to quickly and efficiently add additional connections. However, for maximum performance and responsiveness to peak demand, we recommend not setting this value and instead letting HikariCP act as a fixed-size connection pool. Default: same as maximumPoolSize.
maximumPoolSize	This property controls the maximum size the pool can reach, including both idle and in-use connections. Basically, this value determines the upper limit of actual connections to the database backend. A reasonable value is best determined by your execution environment. When the pool reaches this size and no idle connections are available, calls to getConnection() will block until timeout after connectionTimeout milliseconds. Default: 10
maxLifetime	This property controls the maximum lifetime of connections in the pool. A connection in use will never be retired — it is only removed when closed. To avoid mass connection eviction in the pool, this property applies a slight negative attenuation to each connection. We strongly recommend setting this value, and it should be a few seconds shorter than any database or infrastructure-imposed connection time limit. A value of 0 means no maximum lifetime (infinite lifetime), subject to idleTimeout constraints. Minimum allowed: 30000ms (30 seconds). Default: 1800000 (30 minutes).
idleTimeout	This property controls the maximum time a connection is allowed to sit idle in the pool. This setting only applies when minimumIdle is defined as less than maximumPoolSize. Once the pool reaches minimumIdle connections, idle connections are not retired. Whether a connection is considered idle and retired has a maximum variation of +30 seconds, with an average variation of +15 seconds. A connection is never considered idle and retired before this timeout. A value of 0 means idle connections are never removed from the pool. Minimum allowed: 10000ms (10 seconds). Default: 600000 (10 minutes).
keepaliveTime	This property controls how frequently HikariCP will attempt to keep a connection alive to prevent it from timing out due to database or network infrastructure. This value must be less than maxLifetime. The “keepalive” operation only occurs on idle connections. Minimum allowed: 30000ms (30 seconds), but the ideal value is in the range of a few minutes. Default: 0 (disabled).

The keepaliveTime parameter should be set lower than the database idle connection timeout, TCP idle connection timeout, and all other infrastructure idle timeouts. For PostgreSQL, HikariCP’s keepaliveTime should be set to less than PG’s idle_in_transaction_session_timeout.

Clearly, maximumPoolSize represents the maximum number of connections to the database. Of course, in general, the actual number of connections in the database won’t always stay at maximumPoolSize because the application can’t run at peak load from start to finish. Even after a request peak passes, those idle connections should be released after some time according to idleTimeout or maxLifetime settings. To ensure database availability, this value should be set somewhat lower than the database’s maximum connections. For PostgreSQL, maximumPoolSize should be set to less than PG’s max_connections. There’s room for tuning this parameter, which we’ll discuss below.

minimumIdle is the minimum number of idle connections. For example, if minimumIdle=100 and the database has 10 active sessions, theoretically the total connections in the database should be 100+10. Due to possible connection storms, the actual database connections might be slightly more than active+minimumIdle, but certainly less than maximumPoolSize.

Why are database connections far greater than minimumIdle?

Theoretically, total database connections should only be slightly more than minimumIdle. However, from my actual observation of multi-node connection pool scenarios, even with only 10+ active connections, total database connections far exceed minimumIdle. Observing min(backend_start) and min(state_change) in pg_stat_activity, they stay around maxLifetime, indicating that connection recycling is working. It seems new requests always prefer to establish new connections rather than reuse existing idle ones. Personally, I suspect multi-node deployment is one reason — each node has a low minimumIdle, and some component nodes may have more requests, with instantaneous request counts exceeding minimumIdle, thus creating new connections. Second, it’s related to the maxLifetime parameter — maxLifetime’s purpose is to rotate connections, releasing those constantly in use. This means used connections need time to be released and ideally shouldn’t be reused to avoid extending the release cycle.

Connection Pool Sizing
#

Impact of Excessive Connections
#

In the database world, “as the number of database connections increases, database performance always degrades to some extent.”

For example, Oracle’s connection count impact on performance — refer to this video. With unchanged resource configuration and JDBC concurrency, reducing connections from 2048 to 1024 halved the request response time; reducing to 96 connections dropped response time by tens of times!

What’s the Right Number of Connections?
#

Unless you have a database server that has 1000 cores, it is very unlikely that you really want a maximumPoolSize of 2000.

Unless your database has 1000 cores, you shouldn’t have 2000 connections.

At the most basic level, the database connection count should be set to the number of CPU cores — this achieves maximum CPU performance mode. But this isn’t the full picture. Database consumption isn’t just on CPU, but also on disk and network (memory too, but with relatively less impact). For example, disk reads/writes also take time, and the CPU must wait for disk data to return before proceeding. During I/O wait periods (which can be quite long), it’s better for the CPU not to be idle but to serve other processes. Therefore, based on waiting times for disk and other devices, the database connection count should ideally be higher than the number of CPU cores.

Due to SSD and other disk performance improvements, disk access is now very fast — meaning I/O wait times have decreased, implying connection counts should be tuned even lower.

Tuning too low fails to fully utilize CPU; tuning too high degrades database performance. So what’s the right number? HikariCP provides this formula:

connections = ((core_count * 2) + effective_spindle_count)

Where core_count should not count hyperthreading; effective_spindle_count is the spindle count — if the active dataset is fully cached, effective_spindle_count is zero; as cache hit rate decreases, it should approach the actual spindle count. There’s no established formula for SSDs yet, but it’s certainly less than the above maximum. Of course, these are all theoretical values — real-world situations are more complex, e.g., long connection issues. See About Pool Sizing for details.

Even with 10,000 frontend users, the connection pool cannot be 10,000 — even 1,000 is too many. A smaller connection count, with remaining requests waiting in the pool queue, is the best way to maximize database and CPU performance. See the formula above for connection count settings.

Fixed Pool
#

Fixed pool is a concept advocated by HikariCP’s author Brett Wooldridge to solve the connection storm problem. The concept is already mentioned in the minimumIdle parameter description:

For maximum performance and responsiveness to peak demand, we recommend not setting minimumIdle and instead letting HikariCP act as a fixed-size connection pool. Default: same as maximumPoolSize.

Setting minimumIdle=maximumPoolSize creates a fixed-size connection pool. minimumIdle’s default value equals maximumPoolSize.

As early as 2014, Brett Wooldridge mentioned this concept — see the PG community mailing list. This passage is important, so I’ll translate it verbatim:

In my experience, even pools that maintain a minimum number of idle connections are problematic in responding to burst demand. If you have a pool with a maximum of 30 connections and a target of 10 minimum idle connections, a burst demand requiring 20 connections means the pool can immediately satisfy 10, but then must try to establish another 10 connections before the application’s connection request reaches connectionTimeout. This in turn creates burst demand on the database, slowing down not just connection establishment itself but also transactions that might actually be returning connections to the pool.

Now, if your peak is 100 connections and your median is 50, this doesn’t matter. But I know many workloads where the peak is 1000 and the median is 25 — in such cases you’d want to gradually reduce idle connections.

Ultimately, we adopted a maxPoolSize + minIdle model, where by default they are equal (fixed pool).

While I don’t doubt that such workloads (1000 active connections) exist, if someone is actually doing this, I’d love to hear their reasoning. Unless they have over 128 CPU cores and solid-state storage, they’re basically wasting effort.

This also means that even if the pool size is fixed, you want to rotate actual sessions in and out so they don’t hang onto maximum virtual memory indefinitely.

We do this with a maxLifeTime setting to rotate these connections.

In real scenarios, fixed pool’s protection against connection storm impact is visible. Under fixed pool, when the database’s instantaneous active connections spike, the idle connection count drops but the total connection count remains unchanged, and request response time is minimally affected. If maximumPoolSize is set to a value higher than minimumIdle, a connection storm can cause many new sessions to be created instantly, and new session creation is very resource-intensive — this significantly increases request response time.

Connection Leak Case Study
#

Since I’m not a connection pool expert, I’ll just summarize some recently found connection leak information here.

Connection leaks exhibit the following symptoms:

“Connection is not available” exception. Connection leaks, pool saturation, or the database being overwhelmed by excessive active sessions — new requests error out after exceeding connectionTimeout.
Growth of active connections. Database monitoring clearly shows an increase in active sessions.
Application logs. Application logs also show many connection requests, including active session information.
Database views and logs. pg_stat_activity shows all session states and specific SQL, and logs show new connection authentication information.
HikariCP leak detection. Requires enabling leakDetectionThreshold. HikariCP can detect connection leaks — this parameter is off by default.

For locating connection leaks, you should:

Check application logs, especially around the time the problem first occurred.
Have a proper monitoring system.
Be proficient with debug, trace, and other HikariCP settings.
Set the leakDetectionThreshold parameter.

Possible causes:

Misuse of streaming responses;
Misuse of raw connections;
Prolonged operations within @Transactional method (such as network invocation).
Configuration errors, reference
Virtual threads, reference

References
#

https://github.com/brettwooldridge/HikariCP

https://github.com/brettwooldridge/HikariCP/issues/2148

https://github.com/brettwooldridge/HikariCP/wiki/About-Pool-Sizing

https://blogs.oracle.com/opal/post/always-use-connection-pools

https://mkyong.com/jdbc/hikaripool-1-connection-is-not-available-request-timed-out-after-30002ms/

https://medium.com/@eremeykin/how-to-deal-with-hikaricp-connection-leaks-part-1-1eddc135b464

https://medium.com/@eremeykin/how-to-deal-with-hikaricp-connection-leaks-part-2-847a9629627f

History of Transactions and SSI — PostgreSQL Database Technology Summit Chengdu Stop Sharing

Mon, 12 Aug 2024 00:00:00 +0000

Preface
#

PostgreSQL Database Technology Summit Chengdu Stop
#

Recently (June 17, 2023), the “PostgreSQL Database Technology Summit Chengdu Stop” organized by the PostgreSQL branch of the China Open Source Software Promotion Alliance was successfully held. I had the honor of participating as a speaker and gained a lot from it.

(Summit review and all PPT downloads: PPT downloads are here | PostgreSQL Technology Summit Chengdu Stop Review)

My Sharing
#

My technical sharing topic was: Database History and SSI. I’ve noticed that many domestic technical blogs describe transactions inaccurately, which can confuse beginners. Additionally, many colleagues aren’t very familiar with transaction history and SSI in PostgreSQL. This time, I collected and summarized accurate definitions of transactions, transaction history, and SSI theoretical foundations from Wikipedia, official SQL standards, and various papers. The main thread of the sharing goes from transaction history to anomalies not present in the SQL-92 standard, to how these anomalies can be eliminated, gradually progressing to how SSI is implemented in PostgreSQL. The entire sharing is divided into 4 parts: Transaction Fundamentals, Transaction History, SSI Theoretical Knowledge, and SSI in PostgreSQL.

Transaction Fundamentals
#

Before understanding transaction history and SSI, let’s review and revisit some basic transaction knowledge. The entire chapter will revolve around discussing transactions, and basic transaction knowledge will lead into the problems in transaction history.

What is a Transaction?
#

Original meaning of transaction: A transaction is an exchange, a deal. Exchange is the original meaning of transaction, and what we call transactions in databases comes from this word. Database transaction: A transaction is the basic unit of work in a relational database. For example: Deleting data from table A and inserting data into table B — we can wrap these two actions into one transaction. Both must complete. But due to unexpected factors, the transaction might fail or be canceled halfway through execution. In that case, all operations in the entire transaction must roll back to the state before the transaction — A doesn’t delete and B doesn’t insert.

ACID
#

ACID is an important characteristic of database transactions. It determines whether a transaction is reliable and trustworthy.

Atomicity: All operations within a transaction either complete entirely or cancel entirely. Like atoms in chemistry — indivisible and unsplittable. If a transaction encounters a problem midway and fails to execute, the entire transaction must roll back. Consistency: When a transaction completes, all data remains in a consistent state. This definition is actually somewhat vague. Transactions generally operate on data, and the state of data in the database gets updated. Due to transaction operations, data transitions from one state to another. This state must be reasonable and legitimate — the data logic must be consistent with real-world logic. This might be abstract, so here’s an example: Say A has 100 yuan, B has 200 yuan, their combined total is 300 yuan. Now B transfers 100 yuan to A. Then A has 200 yuan, B has 100 yuan, and their combined total is still 300 yuan. Key point: The data changes in this virtual world should remain consistent with real-world logic. Isolation: The result of executing multiple transactions concurrently must be the same as executing them separately one after another. For example, with 2 transactions, executing them serially one after another must produce the same result as executing them in parallel. (This is the official understanding from Wikipedia and the definition in the SQL standard — please remember this definition, as it’s the focus of this article.) Durability: After a transaction completes, changes to data are permanent. If updated data is placed in memory and disappears when the machine powers off, then it should go to disk. But is disk storage safe? What if the disk fails? We could have a high-availability architecture writing multiple copies of data. Extending further, we could have geographic-level disaster recovery. But if we push further — what if multiple regions all fail? From an architectural perspective, this question seems to have no answer. But from the user’s perspective, it’s actually easier to understand. For example, when a user deposits money — they put the cash in, and their account should display that amount. This number is permanent for the user. The user believes that even if the sky falls, their account should have this number. That is the meaning of durability.

ANSI SQL-92 Standard
#

In 1992, the American National Standards Institute ANSI SQL-92 standard defined 4 isolation levels and 3 anomaly phenomena. Although the database industry today mostly follows ISO international standards, this 1992 American standard had a huge impact on the database industry. I believe many database practitioners are familiar with the 4 isolation levels.

Isolation Levels in the SQL-92 Standard
#

ANSI SQL-92 defines 4 isolation levels: Transaction isolation levels from high to low. Notice Serializable: when all transactions in the system execute in parallel, there is no difference from executing them serially — transactions do not affect each other. Doesn’t this resemble the definition of Isolation in ACID? All 4 isolation levels can satisfy all-or-nothing execution of transactions. They only differ in their definitions of isolation. All isolation levels can have atomicity, consistency, and durability, but different isolation levels have different isolation characteristics. By definition, only Serializable fully satisfies ACID.

Anomaly Phenomena in the SQL-92 Standard
#

The SQL-92 standard defines 3 anomaly phenomena. There are many definitions online, but many are not entirely accurate. Here we directly extract the definitions of the 3 anomaly phenomena from the SQL-92 standard document:

Dirty Read: Transaction T1 updates a row. Transaction T2 can read this row before T1 commits. If T1 executes a rollback, T2 will have read a row that was never committed. Dirty reads have an obvious problem — the user may not know whether the money has actually arrived. Before the transaction completes, the user can query and see money transferred into the account, but if the transaction fails and rolls back for some reason, the money disappears again. This is hard for users to understand.

Non-repeatable Read: Transaction T1 reads a row. Transaction T2 updates or deletes that row and commits. If T1 reads that row again, it will find the row has been changed or deleted. Phantom Read: Transaction T1 reads N rows matching certain conditions. Transaction T2 executes SQL that generates rows satisfying these conditions. When T1 reads again, it finds inconsistent row results. The difference between non-repeatable read and phantom read is: one is caused by other transactions updating or deleting leading to inconsistent reads within the same transaction; the other is caused by other transactions inserting leading to inconsistent reads within the same transaction.

SQL-92 Standard and PostgreSQL
#

In the SQL-92 standard, isolation levels and anomaly phenomena have a stepped relationship. Except for Serializable which has no anomalies, each isolation level adds anomaly phenomena step by step. Now let’s look at the following table — this is the isolation levels and anomaly phenomena in PostgreSQL, which is different from the SQL-92 standard.

Why is PostgreSQL’s isolation level inconsistent with the SQL-92 standard?
#

Why is Read Uncommitted inconsistent with the SQL-92 standard? Read Uncommitted is simply too strange. In relational databases, it’s hard to imagine a scenario for using Read Uncommitted. It severely violates transaction isolation. PostgreSQL treats “Read Uncommitted” as “Read Committed.”
Why is Repeatable Read inconsistent with the SQL-92 standard? PostgreSQL implements MVCC (Multi-Version Concurrency Control) through snapshots. The Repeatable Read level in PostgreSQL is actually the Snapshot Isolation level, which doesn’t have the Phantom Read anomaly.
Although the SQL-92 standard has far-reaching influence, many databases haven’t fully implemented it.
The ANSI SQL-92 standard has vague definitions. The SQL-92 standard is very representative in the database industry — “It’s good, but not good enough.”

Transaction History
#

History of Transactions
#

To understand “It’s good, but not good enough,” we need to review transaction history, going back 40 years. Notice the timing of the SQL-92 standard and the “Critique of SQL-92.” Although the SQL-92 standard was “flawed,” it still had a profound impact on the database industry. Subsequently, after many serializability theories were proven, PostgreSQL became the first commercial database to implement SSI.

Critique of the SQL-92 Standard
#

Shortly after the SQL-92 standard was released, some Microsoft engineers and academics critiqued it and proposed more isolation levels and anomaly phenomena. Where the SQL-92 standard defined 4 isolation levels and 3 anomaly phenomena, the “Critique of SQL-92” had 6 isolation levels and 8 anomaly phenomena.

More isolation levels and anomaly phenomena appeared — they were not defined in ANSI SQL-92.
Snapshot Isolation sits between Repeatable Read and Serializable. This is also one of the reasons why PostgreSQL’s Repeatable Read and Serializable look so similar.
The Write Skew anomaly was identified. It occurs at the Snapshot Isolation level.

Isolation Levels of Popular Databases
#

MySQL at Serializable isolation level: reads acquire shared read locks on data, meaning reads block writes.
Oracle can also set the Serializable isolation level and claims to support serializability, but it’s not true serializability — it’s just Snapshot Isolation.
PostgreSQL supports Serializable. It implements serializability on top of Snapshot Isolation, fully named Serializable Snapshot Isolation (SSI), where reads and writes do not block each other.

You can see the differences among the three — only PostgreSQL’s Serializable has real substance.

Why Did Oracle Deceive Us?
#

What did Oracle deceive us about? It passed off the Snapshot Isolation isolation level as the Serializable isolation level. Why did this happen? If we add Snapshot Isolation to the ANSI SQL-92 standard:

The SQL-92 standard defines fewer anomaly phenomena and doesn’t define Snapshot Isolation. By the SQL-92 standard’s view, Snapshot Isolation looks similar to Serializable.
Most relational databases follow the SQL-92 standard, including Oracle. But when better standards later emerged, they didn’t make changes.

Why Do Weak Isolation Levels Have Academic Problems but Few Serious Real-World Issues?
#

Anomaly phenomena at non-serializable isolation levels generally require high concurrency to manifest. Low-concurrency databases are unlikely to encounter problems.
When anomaly phenomena do occur, some applications may not notice them, or may detect anomalies but find them unimportant.
Data might be anomalous, but the application simply returns an error and enters an anomaly handling routine.
Costs are too high. Not only is the development cost of database serializable isolation levels high, but applications also need adaptation costs for serializability. Just understanding this complex theory is no easy task.
High-level isolation loses some performance. Extensive modification work may be thankless — applications need to choose between “high concurrency” and “no anomaly phenomena.”
Businesses develop based on mechanisms rather than rules. Businesses somewhat adapt to the anomaly phenomena of weak isolation levels, especially Read Committed.

What’s the Point of Serializable?
#

If weak isolation seems to work fine in the real world, what’s the point of Serializable? There is actually a point:

Although applications adapt to weak isolation levels, it doesn’t mean they truly understand them.
Using Serializable, applications can greatly reduce concerns about data anomalies.
Except for Serializable, all other isolation levels have their own anomaly phenomena and don’t fully satisfy ACID’s Isolation property.
Serializable can eliminate anomaly phenomena — the “termites” — fully ensuring data safety.
Serializable has been proven theoretically achievable.
Some serializable implementations do significantly reduce concurrency, but there are other implementations with minimal concurrency impact. For example, Serializable Snapshot Isolation (SSI).

SSI Theoretical Knowledge
#

After all that about transaction fundamentals and history, we finally arrive at the concept of SSI. But before understanding SSI, we need to understand two more concepts: Serializable and Snapshot Isolation.

Serializable
#

Meaning of Serializable If each transaction itself is correct (satisfying certain integrity conditions), then any serial schedule including these transactions is correct (its transactions still satisfy their conditions): “Serial” means transactions don’t overlap in time and cannot interfere with each other — i.e., there exists complete isolation between them.

Implementation of Serializable In early transaction development, Serializable was implemented through Strict Two-Phase Locking (S2PL), where reads and writes block each other until the transaction ends. This eliminated anomaly phenomena but S2PL lost high performance. Besides S2PL, there are other ways to achieve serializability, such as Serializable Snapshot Isolation (SSI).

Significance of Serializable To ensure no anomalies, Serializable sacrifices some concurrency (varying by implementation approach), but it truly guarantees ACID isolation for data. That is to say, databases that haven’t implemented serializability don’t fully support ACID properties. Serializable has been proven theoretically achievable, but the real database world is somewhat “abnormal.” In practice, Serializable is the highest transaction isolation level and is strongly recommended by academics and industry leaders, yet the vast majority of databases run at Read Committed or Snapshot Isolation levels.

Snapshot Isolation
#

Definition of Snapshot Isolation Transactions executing under Snapshot Isolation operate on a snapshot of the database taken at the start of the transaction. When the transaction ends, it will only commit successfully if the values it updated haven’t been externally changed since the snapshot was taken. As the name implies, Snapshot Isolation uses snapshots, which are widely used to implement MVCC, enabling multi-version concurrency mechanisms to support concurrent transaction execution by users.

Emergence of Snapshot Isolation ANSI SQL-92 did not define Snapshot Isolation (SI). This isolation level emerged as the database industry evolved. The 1992 ANSI SQL-92 standard was defined based on database locks, so there was no definition for the Snapshot Isolation level. It wasn’t proposed until the 1995 “Critique” appeared.

SSI
#

Serializable Snapshot Isolation (SSI) Given the widespread use of Snapshot Isolation and the academic goal that databases should achieve the Serializable isolation level, Serializable Snapshot Isolation (SSI), as the name suggests, implements serializability on top of Snapshot Isolation.

Why SSI? Due to the vagueness of the ANSI SQL-92 standard, although it didn’t define Snapshot Isolation, many databases actually use it. And Snapshot Isolation also has some anomaly phenomena (including Write Skew). SSI emerged to address these anomaly phenomena.

Advantages of SSI over S2PL Traditional serializability is implemented through S2PL. Under S2PL, write operations block other transactions’ reads and writes. Although it achieves serializability without Write Skew anomalies, it generates many lock conflicts, reducing concurrency performance. In contrast, MVCC implemented through snapshots has non-blocking reads and writes, with only write-write conflicts. SSI built on this foundation has much less impact on concurrency compared to traditional S2PL.

PostgreSQL Implements SSI PostgreSQL began implementing SSI in version 9.1, becoming the first commercial database to implement SSI.

Three Types of Dependencies
#

Read-Write Dependency (wr): Transaction T1 writes a version of a data item, and transaction T2 reads this version, meaning T1 precedes T2. Write-Write Dependency (ww): Transaction T1 writes a version of a data item, and transaction T2 replaces this version with a new one, meaning T1 precedes T2. Read-Write Anti-dependency (rw): Transaction T1 writes a version of a data item, and transaction T2 reads the version before this one, meaning T2 precedes T1.

Write Skew Theory
#

When certain conflicts form a cycle, serialization anomalies occur. That is to say, some concurrently executing transactions are theoretically non-serializable. One of the more easily understood examples is Write Skew. Write skew only occurs in the rw model — ww and wr won’t cause write skew — and transactions must be under concurrent conditions for it to appear.

Simple Write Skew: Transaction T1 has an rw anti-dependency on T2, and T2 also has an rw anti-dependency on T1. The concurrent execution of these two transactions is non-serializable.

Real-World Write Skew Problems
#

Many real-world cases can produce Write Skew anomalies. Let’s use the classic black-and-white ball problem to understand Write Skew: There are 4 balls in a bag: 2 white and 2 black. Now there are two transactions, P and Q. P changes all black balls to white, Q changes all white balls to black. There can be two serial executions: <P, Q> or <Q, P>. In both cases, the final result is 4 white balls or 4 black balls. However, Snapshot Isolation allows another result: Transaction P takes out 2 black balls Transaction Q takes out 2 white balls Transaction P changes all black balls in hand to white and puts them back Transaction Q changes all white balls in hand to black and puts them back Now the bag still has 2 black balls and 2 white balls. This is impossible in any serial execution. But this is valid under Snapshot Isolation: each transaction maintains a consistent view of the database, and its write set doesn’t overlap with any concurrent transaction’s write set, resulting in the white and black balls exchanging.

We can also make the problem more concrete and practical. Here’s a rough example: Suppose I have several bank cards, half frozen and half unfrozen. At one terminal, I execute freezing all cards. At another terminal, I immediately execute unfreezing all cards. From an intent perspective, my cards should all be unfrozen. But a strange phenomenon occurs: previously frozen cards become unfrozen, and previously unfrozen cards become frozen. As a customer, I would be confused.

The black-and-white ball problem illustrates: Snapshot Isolation execution results are inconsistent with Serializable execution results. Under Snapshot Isolation, a Write Skew anomaly occurs, and data results don’t match expectations.

SSI in PostgreSQL
#

How PostgreSQL Handles SSI
#

It’s actually simple — cancel the pivot transaction that forms the “dangerous structure.” We first set the isolation level to Serializable for both. The table has some white balls and some black balls.

T1	T2
set default_transaction_isolation = ‘serializable’;	set default_transaction_isolation = ‘serializable’;
begin; update dots set color = ‘black’ where color = ‘white’;
	begin; update dots set color = ‘white’ where color = ‘black’;
commit;
	commit;
	`ERROR: could not serialize access due to read/write dependencies among transactions DETAIL: Reason code: Canceled on identification as a pivot, during commit attempt. HINT: The transaction might succeed if retried.`

Transaction 1 changes all white to black, Transaction 2 changes all black to white, then both commit. The first transaction to commit succeeds, the second fails. The error says: could not serialize access due to read/write dependencies among transactions, canceled on identification as a pivot. If you retry the transaction, it might succeed. Of course it would succeed here — the other transaction has already completed, so one transaction alone cannot form a dependency cycle. At other isolation levels like Repeatable Read or Read Committed, these two transactions would execute without any error, running normally, but the data results would differ from SSI’s results.

PostgreSQL SSI Implementation Optimizations
#

PostgreSQL implements Serializable SSI on top of Snapshot Isolation and has made many optimizations to improve concurrency at high isolation levels. PostgreSQL’s SSI optimizations mainly include 3 points: Safe Snapshots: Read-only transactions that won’t create cyclic structures don’t need conflict detection, reducing checking overhead and memory burden.

Deferrable Transactions: Deferrable transactions can be retried. When a “dangerous structure” is detected, the deferrable transaction is canceled and then attempted again. Deferrable transactions need to be explicitly declared.

Detection Granularity Escalation: Multiple fine-grained locks can be combined into coarse-grained locks to reduce memory overhead.

Optimization Results — Performance Benchmark Comparison: The green line is the Snapshot Isolation baseline. The blue line shows PostgreSQL’s SSI performance, which is already very close to Snapshot Isolation. The brown line is SSI without read-only transactions — all data-changing transactions — showing how much read-only transaction optimization improves performance. In typical business systems, read-only transactions outnumber change transactions. The red line is serializability implemented through Strict Two-Phase Locking — the performance is abysmal.

The table below shows concurrency pressure and transaction failure rates. Since some transactions need to be canceled to break cycles, Serializable inevitably cancels more transactions than weak isolation. This table also shows that PostgreSQL’s SSI has far higher concurrency and transaction success rates than Strict Two-Phase Locking.

Optimization Results — Request Volume and Failure Rate:

Summary
#

Serializable can simplify system development problems. Developers don’t need to worry about transaction anomalies under concurrency, especially in today’s increasingly high-concurrency systems.
PostgreSQL’s Serializable is clearly better than the Strict Two-Phase Locking model. Not only better performance, but also lower transaction abort probability.
PostgreSQL is the first commercial database to implement SSI, while many traditional relational databases don’t support serializability at all. PostgreSQL has taken a big step forward.
PostgreSQL not only implemented SSI but also made many optimizations on top of it, such as read-only transaction and memory optimizations, with significant results.

How to Solve Index Split Contention?

Mon, 12 Aug 2024 00:00:00 +0000

Index Splitting
#

When an index block is nearly full, index splitting occurs. Index splitting comes in two forms: 55 and 91:

The difference between index splitting and the enq: TX - index contention wait event Whether 55 or 91 splitting, both are normal index behavior as data volume increases. Index splitting is a normal phenomenon caused by growing data volume leading to larger indexes — when an index can’t hold more data, it naturally needs more index blocks. There are hardly any scenarios with tables but no indexes (only during initial data loading would one consider inserting data first and building indexes afterward). Although index splitting consumes some resources, in today’s Oracle environments it can complete quickly. Only when there are too many indexes does it affect insert efficiency.

However, the enq: TX - index contention wait is NOT normal. enq: TX - index contention indicates that SQL statements are waiting on an index block that is currently being split. Essentially, DML concurrency is too high and all sessions are waiting on the splitting index block.

Why does enq: TX - index contention always occur on sequentially inserted columns? Although both 55 and 55 splits are possible in real scenarios, enq: TX - index contention frequently occurs with 91 splits. This is because columns like sequences and timestamps usually have indexes, and sequential inserts are common. The rightmost block is always the hot block, and subsequent inserts must wait for the split block to complete before they can proceed — this causes enq: TX - index contention. Why don’t UUID indexes cause enq: TX - index contention? Because UUID indexes are unordered — inserting causes UUID index splits, but it’s unlikely that subsequent UUID values also land on that same splitting index block. So UUID has index splitting but doesn’t form an enq wait queue leading to enq: TX - index contention.

Solutions
#

Note: what we need to solve is the index split wait enq: TX - index contention, not index splitting itself. Solutions:

1. Reverse Index A reverse index stores key values in the opposite order. For example, for the value ‘1111 0001’, a normal index places it after ‘0000 0002’; with a reverse index, it’s placed before ‘0000 0002’. Think about a timestamp column — normally it’s a rightmost hot spot. After reversing, seconds, minutes, and hours sort first. One index block might contain data from different months but the same second. This way, the rightmost hot block essentially disappears — reverse indexes scatter hot spots across various index blocks. Limitations: Requires index modification; may lose index range scan capability. Sequentially growing columns cannot use index range scans (e.g., timestamp columns). In some scenarios, reverse key values might still work — requires specific analysis. Syntax:

CREATE INDEX reveridx ON tablzl (name) REVERSE;

2. Hash-Partitioned Index Creating a hash-partitioned index on a regular table is equivalent to keeping the table unchanged but partitioning the index, thus scattering the rightmost hot block across partitions. For example, an 8-partition hash-partitioned index divides the index into 8 segments, creating 8 rightmost hot spots and alleviating the index split problem. Limitations: Requires index modification; affects index range query performance — requires balancing insert hot spot mitigation vs. query efficiency. Equality and IN queries can efficiently use hash-partitioned indexes. From the official documentation:

Queries involving equality and IN predicates on index partitioning key can efficiently use global hash partitioned index to answer queries quickly

However, range scan efficiency decreases — the more partitions, the greater the decrease (though more partitions also provide better hot spot relief). This is clearly a balancing act. Tests show that with 8 partitions, logical reads for range scans increase nearly 8x. After partitioning, indexes within each partition remain ordered, and clustering factor differences are minor — the cost of scanning the index is similar, but the cost of table access increases. If a regular index has 8 entries in one block pointing to 1 data block (1 logical read), after hash partitioning across 8 partitions (1 index block each), it becomes 8 logical reads. This is why range scan index performance degrades. Syntax:

CREATE INDEX cust_last_name_ix ON customers (cust_last_name)
 GLOBAL PARTITION BY HASH (cust_last_name)
 PARTITIONS 4;

3. Using Table Partitioning to Scatter Indexes Partition the table and create local indexes to scatter the rightmost hot spots. Limitations: The partition key cannot be the index column (otherwise it defeats the purpose); requires table modification; if existing SQL already has partition key predicates, range scan efficiency is not affected.

4. Reduce Concurrency Reducing concurrency is the ultimate weapon. Index split contention is fundamentally caused by excessively high concurrency — generally, without dozens of concurrent inserts, index split contention won’t occur.

5. Modify Index Block Size Place index blocks in 16K or 32K tablespaces. In theory, this should help because indexes can hold more data and splitting occurs less frequently. However, performance testing is needed, and other parameters may need adjustment.

6. Remove the Index Removing the index is also an option. Based on business requirements, if the index is not important, drop it. Or use range queries with partitioned tables, leveraging partition pruning instead of indexes.

Why These Approaches Don’t Work???
#

Increasing ITL transaction slots: Index block transaction slots may also be insufficient under high concurrency — this is indeed similar to index splitting, but the wait event is enq: TX - allocate ITL entry. If this wait is observed and traced to index blocks, it indicates high concurrency on the index. Reverse indexes and hash-partitioned indexes can also help, and adjusting initrans may solve the problem. However, the root causes of these two wait events differ — index splitting doesn’t always come with transaction slot issues.
Adjusting index block PCTFREE: PCTFREE indicates that when a block’s free space falls below PCTFREE, it is no longer recorded in FREELIST and cannot accept new inserts. Consider two cases: increasing and decreasing PCTFREE. Increasing PCTFREE only worsens index splitting. Decreasing PCTFREE seems effective — similar to adjusting block size in principle — but in real scenarios PCTFREE defaults to 10%, which is already hard to reduce further, so the effect is negligible.
Rebuilding indexes to reduce fragmentation: This is essentially unrelated — it doesn’t solve the rightmost hot block problem.

References
#

https://blog.csdn.net/lihuarongaini/article/details/101299328 https://docs.oracle.com/cd/E11882_01/server.112/e41573/data_acc.htm#PFGRF94786

Acknowledgments: 豪桑, 用哥

My 2023 Year-End Summary

Mon, 12 Aug 2024 00:00:00 +0000

As a DBA
#

Since early 2023, I set my main task for the year — learn the PostgreSQL database. Though I didn’t set detailed plans, the overall goal was to finish learning some foundational PostgreSQL knowledge. Later I found I had oversimplified things — the cost of learning PostgreSQL was far greater than I imagined, and I didn’t achieve this goal in 2023. For example, the PostgreSQL transaction chapter: I thought I could finish it in 2 weeks, but it took me about 2 months. Regardless, persistent learning did yield some results:

Among them, the optimizer chapter was actually not completed. Though I’m guilty, I still need to explain. The optimization chapter has been in progress for over two months — not because I was slacking off, but because it’s simply impossible to finish. It has already reached Typora’s text limit — around 8000 characters it starts lagging, so I had to passively split it into parts. It’s already split to Part 4:

Even so, the optimization chapter is probably less than half done. I can only shamelessly carry it over to the next year… Personally, I think another 4 months should let me complete the optimization chapter… Even then, the priority needs to be pushed back — there’s really not enough time!

READING
#

My main profession is databases, so I should spend time on databases, and extracurricular reading should take a back seat. However, I still don’t want to give up this part, for three reasons I think:

The value brought by reading is immeasurable in the short term
Reading brings a pleasant sense of intellectual enrichment
I use fragmented time to read, only spending 2-3 hours writing reading notes, which doesn’t take up too much study time

I’ve certainly read some PostgreSQL technical books, but I read them with a targeted approach. For example, for optimization, I’d bring together “The Internals of PostgreSQL,” “PostgreSQL Technical Internals: Query Optimization Deep Dive,” “PostgreSQL Query Engine Source Code Technical Analysis,” and “The Art of Database Query Optimizer” to study a particular knowledge point together. I wasn’t focused on whether I’d finish them, and I didn’t read them cover-to-cover in order. So the reading list here only covers extracurricular books.

2023 Extracurricular Reading List (ranked by preference):

“Homo Deus”
“Romance of the Three Kingdoms”
The “Space Odyssey” series: 2001, 2010, 2060, 3001
“Elon Musk”
“Chimpanzee Politics”
“Goodbye, the Age of Mediocrity”
“Wild”
“Are We Smart Enough to Know How Smart Animals Are?”
“To Kill a Mockingbird”
“Rich Dad Poor Dad”
“When Breath Becomes Air”
“The Metamorphosis,” “The Judgment,” “A Hunger Artist” and other Kafka short stories

Not great: “What Life Could Mean to You,” “How to Win Friends and Influence People,” “The Courage to Be Disliked”

Blog and WeChat Official Account
#

I publish articles through two channels:

CSDN Blog: https://liuzhilong.blog.csdn.net
WeChat Official Account: liuzhilong62

I’ve been persisting with blogging for many years. The big change in 2023 was mainly writing about PostgreSQL and increasing technical depth. The WeChat Official Account is a new venture I started this year, and it was a major experiment in 2023. Both blogs and official accounts can be used for technical sharing, but their audiences are somewhat different. A blog can serve as a technical accumulation, while an official account is more like a technical news feed. There are many big names in the community who publish daily (even multiple times a day) — I greatly admire that. But there are also big names who focus on quality articles without worrying about daily posting. I personally prefer the latter approach — learning a domain’s knowledge roughly in one go, which feels more holistic and targeted. Often I split longer articles into parts for the official account (I don’t even like reading overly long articles myself). On my blog I don’t split them, so readers interested in a particular article can search for it on CSDN — it’s easier to read there.

Why write?

Self-learning value
Technical research value
Dissemination value

The efficiency of active learning far exceeds passive learning, just like this learning pyramid (image from “Rich Dad Poor Dad” — the value of extracurricular reading!):

Opportunities like hands-on practice and presentations are rare and hard to come by. Outputting what you’ve learned as articles greatly improves your understanding of knowledge points. Reading an article might take just ten minutes, but producing it as an article may take more than ten times that long.

This year I also tried doing pure translation-style technical articles. Although the technical research value isn’t high, there’s still learning value and dissemination value. Reading something once versus translating it once leads to different levels of understanding, just like what I said above: active learning. However, what bothers me a bit now is: previously, for things I couldn’t understand, I’d use Google Translate for a rough pass and then polish it myself. Now with GPT, it can translate an entire article and I barely need to change any words or sentences. The active learning value has been severely diluted — the AI is doing all the learning…

My writing style changed significantly in 2023. I wrote about various things and tried everything. Of course, I know one should focus on vertical content, but I still couldn’t resist doing random things — I haven’t even settled on a name for my official account yet. Currently, what’s clear is: technical articles and extracurricular reading notes, with technical articles as the main focus. Other types of articles probably won’t be written anymore. Whether I’ll adjust later, I don’t know. At least the official account still has room for adjustment. Anyway, let’s keep it like this — launch first, adjust later.

2023 blog statistics are hard to track now. I can only provide blog data from 2017 to 2023 as a snapshot.

CSDN Blog:

WeChat Official Account followers:

Final Thoughts
#

The biggest realization of 2023 — time. There’s really not enough time!

On June 17, 2023, I participated in the PostgreSQL Database Technology Summit Chengdu stop and shared my fresh, hot-off-the-press PostgreSQL transaction knowledge with the experts. It was my first time on stage and I was quite nervous. I must thank Boss Can for the opportunity. There was a small episode during this sharing that shows how pressed for time I was in 2023. I also had part-time graduate studies — the day of the sharing was also my final exam day. After finishing my talk, I rushed straight to the airport… In the end, I missed 3 exams and had to retake them… It was too hard.

I’ve completely given up on work-life balance — having a work-learning balance would be good enough. Every day after work I don’t think about resting but about going home to study. In the end, there were still many things unfinished, left to my 2024 self.

Expectations for 2024:

Complete my thesis and graduate smoothly
Finish the PostgreSQL optimization section
We’ll see about the rest

People from Another World

Mon, 12 Aug 2024 00:00:00 +0000

Vacation
#

I took a long vacation and went back to my hometown before my leave days expired — not just to escape the busyness of work, but also to visit my grandparents. For working people like us, going back to our hometown is really difficult. If it’s just a weekend trip, we’d only get one day of rest before having to head back — too exhausting. We don’t get many vacation days to begin with, and when we do, most people think about driving out to see some scenery or just staying home for a few days doing nothing. No one usually thinks of using their precious leave to visit elderly relatives back home.

Ironically, the leave I used to visit my grandparents was childcare leave, not some kind of “eldercare leave.” It seems the world doesn’t have such a thing as “eldercare leave” — only family visit leave. Although there is legally a “family visit leave” provision, never mind that it isn’t specifically designed for visiting the elderly — just look at those impossibly long qualifiers. For the vast majority of people, family visit leave essentially doesn’t exist.

Using childcare leave not to care for children but to visit the elderly — I imagine most people wouldn’t do that. Am I the only oddball who would? Well, at least this is how I see it: raising children and caring for the elderly are equally important; we shouldn’t favor one over the other. Society and working people tend to prioritize the former. Regardless, I still wanted to go back and spend time with them, to see what the old couple does every day, how they live, whether they face any difficulties, and how they cope with those difficulties. So I went back, alone.

The end of the road. The place where the old couple lives is where I grew up. It’s quite hidden — you have to turn off the main road onto a mountain path and go a long way, all the way to the end. It feels like a place cut off from the world. When you arrive there, it’s as if all connection to the outside world ceases to exist.

It’s not actually my ancestral hometown, but I prefer to call it that. It’s a mining area. Because it’s built into the mountainside, the mine has a striking three-dimensional quality — so much so that I’m in awe of the predecessors who designed it. I still don’t quite know how to describe the administrative level of this place. It’s not a village, not a town — more modern than a village but smaller than a town. When I was little I thought this place was huge; now I realize you can walk through the entire mining area in just ten minutes.

The whole place relies on coal mining as its economic pillar. It once prospered, but now it has declined significantly. There are still miners who go underground, but in the living quarters, you no longer see young people like me. The mine has an elementary school; when I attended, there were about 70 students per grade. Now there are only seven.

The childhood memories there are overwhelmingly strong — like a paradise, a sanctuary untouched by worldly strife, another world. Being far from modern society, you only need the basics to get by, and time seems to pass slowly. A place like this is indeed very suitable for retirement — and indeed, there are many elderly people here.

Food
#

When I was little, the market was fairly lively. I remember the poultry vendor would submerge whole chickens in something black and tar-like before plucking them — the poultry area was always filthy. Now the market no longer sells fresh meat; you can only buy vegetables grown by nearby farmers. If you want fresh meat, you have to go to the village market day or take a bus into the city.

Because the old couple is extremely frugal, I was initially worried they lived too simply — maybe just rice and vegetables every day. When I went back this time, I didn’t tell them exactly when I’d arrive. When I got home, I found they had even bought braised duck — I was quite relieved. My return made them very happy, and with just the three of us, they made five or six dishes every day. I even started to wonder if I was there to keep them company or to cause them trouble.

Maybe I’ve been spoiled by the rich flavors of the outside world. At first, when they asked, “Is this dish good?” I couldn’t bring myself to say what I really thought. At moments like this, I recall a line from some book: “Humans cannot directly judge the value of something; only by comparing it to something else do they know its worth.” The same goes for food. When you taste something for the first time, you don’t actually know if it’s good or not. If you do know, it must be because you’ve already compared it to something in your memory. When I was little and first tried hotpot, adults would always ask, “Is this hotpot good?” To be honest, I had no idea — I didn’t even know what “good” was supposed to taste like. I just ate.

Now my palate has indeed grown more demanding, but here, I wanted to reset everything, to press that “restore factory settings” button. I can say with complete sincerity: what they cook is delicious.

One more thing: at one point I offered to wash the dishes. They said, “Put them down, you don’t know how — we wash dishes with rice water. You wouldn’t get them clean. Dish soap is full of chemicals; we don’t use that stuff.”

Traditional Chinese Medicine
#

On this trip, I discovered a fact: elderly people are extremely dependent on medication. Their medicine cabinets are always stuffed with all kinds of drugs — Western medicine, Chinese medicine, cold medicine, anti-inflammatory drugs, ointments, supplements — a whole pile. Whenever they feel something wrong with their body, they reach for whatever they think will help. During my visit, my rhinitis flared up (an old problem of mine) — nonstop sneezing and runny nose. They kept urging me to take cold medicine, recommending Ganmaoling or cephalosporin. I must have said at least ten times: “It’s rhinitis, not a cold.” They, of course, had no idea how to treat this kind of rhinitis, so they just kept urging me to take cold medicine.

One day I took them into the city. Besides a supermarket run, the more important errand was buying medicine. Buying medicine meant both Chinese and Western.

The Chinese medicine was purchased at a Yunnan herbal shop. The shop owner had a buzz cut, a black T-shirt, a silver necklace, and a brown beaded bracelet — he looked quite burly. With his tough-guy appearance, I didn’t even dare to speak loudly to him, though my grandfather didn’t seem to notice any of that. The shop was mostly filled with herbs I couldn’t name, sold by weight, quite expensive — not your typical Chinese medicine. Clearly, my grandfather was a regular customer; the owner knew him. But it seemed my grandfather didn’t really know how to pick herbs either: “Boss, just weigh me 300 yuan’s worth based on my health condition.” So the owner grabbed a bit from here, a bit from there, and finally ground everything into powder.

Maybe I’ve studied too much — I’ve always been skeptical of traditional Chinese medicine, simply because I find it lacks convincing rationale. I was quite worried they’d get scammed; these herbal medicine dealers prey specifically on the elderly. But my grandfather said: “Before, your grandmother had constant headaches. After taking this medicine, the headaches stopped.” So it seemed to work. Western medicine is indeed far too unfriendly to the elderly.

Western Medicine
#

After buying the Chinese medicine, we walked a long way to a pharmacy to buy Western medicine. That pharmacy might be one of the few they know.

The vast majority of drugs in that pharmacy couldn’t be reimbursed. Only in a tiny, shabby room deep inside were a small selection of reimbursable drugs. I looked around and barely recognized any of them — all named with chemical formulas, completely incomprehensible. Only things like Ganmaoling and loquat syrup were familiar. My grandfather fell into the same difficulty choosing. He recognized cephalosporin, but the pharmacy girl said they didn’t have it. He got a bit angry and said to her: “Don’t you have any decent medicine?” (“Decent medicine”? I tried to parse what he meant.) The girl pulled out a red box of nicely packaged health supplements from somewhere. My grandfather couldn’t read the tiny text on the box, so he asked me to read it to him and tell him what it treated. I looked at it — the thing claimed to treat everything — so I didn’t read it and handed it back to the girl. In the end, they only picked up a few common cold and cough remedies.

My grandfather repeatedly told me along the way that he gets 170 yuan of medical insurance reimbursement per year. I could tell he really, really wanted to spend that 170 yuan, to stockpile some medicine at home. That’s why he wanted to go to a Western pharmacy, and that’s why we walked all the way to this pharmacy that accepts insurance reimbursement.

But there was some trouble at checkout. The cashier girl had looked unhappy from the start. She took the medicine and rattled off a bunch of things I didn’t understand — and my grandfather clearly didn’t either. The only thing we caught was: “These can’t be reimbursed.”

The girl said: “There’s a threshold fee of 150 yuan for reimbursement, and you haven’t paid the threshold fee yet.”

My grandfather said: “Is the threshold fee like the 150-yuan bed fee hospitals used to charge?”

The girl paused, then said impatiently: “Yes, yes, whatever you say is right.”

My grandfather got a bit angry: “Forget it, I don’t want them!”

I quickly asked the girl what exactly this threshold fee meant. Without a word, she pointed to a notice posted on the window — a table explaining the threshold fee. I couldn’t quite make sense of it either, but I understood that this threshold fee had to be paid. I thought about it — when I see a doctor, I just swipe my insurance card directly. What’s all this about reimbursement? I was even more confused.

I said: “Can I use my insurance card?”

The girl said: “Out-of-region cards won’t work.”

I said: “Can I just pay with Alipay?”

Seeing me about to pay, my grandfather immediately stopped me: “There’s absolutely no way I’m letting you pay for this.” He pulled cash from his bag and paid. I understood — for me, a hundred-something yuan is nothing, but for them, it’s still money they’re reluctant to part with.

Technology
#

We marvel at how fast technology advances, always bringing new things that change our way of life and make it more convenient. Working people chase technology and immerse themselves in it. But for the elderly, technology is an entirely different story.

In every aspect of elderly people’s lives, things involving technology are exceedingly rare. The most commonly used thing is a phone — a smartphone. They seem to have adapted well to the fast-paced entertainment of apps like Douyin (TikTok), and they also play with their phones watching short videos before bed. (What they use is probably not actual Douyin, but some other app with recommended short videos.)

But that’s about the limit. They don’t really understand how phones work. For example, when they make phone calls — whether it’s my grandmother or grandfather — neither of them hangs up after finishing a call. It’s not that they don’t want to; they just don’t know where to find the hang-up button. If after a call they look at the phone and see a red hang-up button, they’ll press it. But if the screen is locked or the screen has changed, they won’t know how. My grandmother said to me: “Take a look at my phone — after I hang up, why does it keep making noise, keep making noise~” In fact, the call hadn’t ended at all; the screen had just gone dark and she thought it was hung up. If they call someone else, it’s fine, but if they call each other, it could be a disaster — because no one hangs up.

And WeChat messages — they have absolutely no grasp of how WeChat messages work. They don’t know how to find someone’s chat window, don’t know who sent them a message, don’t know where messages go. Later, when we went traveling and I took photos for them, they asked me to put the photos on their phones (meaning in their photo albums). I had to operate both of their phones one by one to download photos from WeChat, making sure the downloaded photos were immediately visible in the album — otherwise, they’d never find them.

Traveling
#

Taking the old couple out to travel was an important mission of this trip. I hadn’t originally planned it — I just wanted to empty my mind, breathe some fresh air, and stay there experiencing the slow passage of time. But they really enjoy going out. As soon as I arrived, my grandfather proactively suggested I could drive them somewhere for fun.

We visited Zhu De’s Former Residence, Langzhong Ancient City, and Nanchong — two days and one night. Traveling with elderly people requires more consideration — they can’t sit in a car too long or walk too much. So we couldn’t really do that many things. But their philosophy of travel is different from ours; they lean more toward “checking in,” valuing the fact that they’ve “been here.” So they absolutely must take photos at landmark spots with the place name written on them~

They also prefer crowded places over scenic spots with few people. In Langzhong, they clearly enjoyed being inside the ancient city — the bustling, noisy, lively atmosphere. They even video-called my aunt and shouted, “We’re in Langzhong!!” (with heavy emphasis), grinning ear to ear. Meanwhile, at White Pagoda Hill (you can drive up, very elderly-friendly), overlooking the panoramic view of Langzhong, I was immersed in a “what a view” moment. My grandmother looked for two minutes, took two photos, and that was it. I said, “Look at the scenery, it’s so beautiful — we came all the way up here.” She replied, “I already looked.”

The Shed
#

Usually, my grandfather goes to the small park to watch others play cards, and my grandmother plays mahjong — no money involved.

Besides cards, the place they spend the most time is the shed downstairs. A few discarded stools and chairs from various families are gathered under the shed, with a stove in the middle where they burn firewood in winter. Everyone upstairs and downstairs knows each other — all grandparent-aged, on very good terms with my grandparents. Neighbors will sit together and chat whenever they’re free. This is the most important social venue for the “neighborhood” (it’s not really a neighborhood, just two buildings).

One evening, I sat in the shed listening to them talk. One grandmother said: “Your grandson is so good, taking leave to come back and keep the elderly company. We all say your grandson is wonderful.” I was a bit embarrassed, but thought — let this evaluation stay in the minds of these elders.

One grandfather said: “I told xx’s family: come back once a month, spend time with the elderly, no need to give them money. What would they use that money for? They can get by just fine. But you’ve grown up and left, and without company for a long time, they feel lonely.” I thought, this old man really understands things. He added: “Once, so-and-so died. His whole family came back for the funeral. They brought him fruit and food — what’s the use? Did he get to eat any of it? To put it bluntly, that was all for show — for us to see. Once a person is gone, none of it matters.” Wow. This old man truly gets it.

Coming back once a month is extremely difficult for working people — it’s just not realistic. Next year I won’t even have this childcare leave anymore. When will I come back next time? I can’t think of an answer. As we pass day after day in relentless busyness at work, how do the elderly pass their days — day after day of idleness and loneliness?

Random Thoughts
#

How should the elderly face death? Every time they mention death, it’s always with a joking tone, but more than that, there’s resignation. How should a person face death? When I’m old and the deadline is approaching, how will I face it?

While chatting in the shed, I couldn’t name any of these grandparent-aged people, but they all remembered me, knew how I grew up. My life seems to be a part of their lives, proof of my existence — even if this memory only lasts for a time. Yet that still has meaning, doesn’t it? The bonds of life exist in this way. There are billions of people in this world, and the vast, vast majority are fleeting meteors — remembered by no one, mentioned in no record.

This society is remarkably unfriendly to the elderly. Social rules are too complex; they struggle to understand phones, healthcare, and insurance systems, so they can only huddle within their own social circles and flee from this incomprehensible society. At the same time, society has developed rapidly in recent years — children have mostly moved away for their own families and careers. For the elderly, they’re happy to see their children thriving, but the distance is vast, and mutual companionship is hard to come by. While society focuses on childcare and increasing birth benefits, no one pays attention to the issue of eldercare and companionship. I doubt there will ever be such a thing as “eldercare leave.”

My grandmother has poor hearing. Even with a hearing aid, it’s only slightly better. Often when I talk to her, she doesn’t follow at all and answers about something else entirely. But I can’t bring myself to raise my voice — it feels so rude. Leaning in close to speak makes her self-conscious. I suggested they come live with us in Chengdu, but she wouldn’t agree under any circumstances. I think maybe it’s because her hearing loss makes her afraid of communicating with people, timid in social situations. Only there, in the mining community, do the neighbors treat her well — it gives her a sense of security. One elderly woman said: “Being hard of hearing is good — it adds years to your life.”

Written — April 2023

The term “Popo” (a Chinese term for grandmother) still exists in my generation, but my children no longer say “Popo” — they say “Nainai” instead. Perhaps “Popo” is the last time this term will be used in our family line — may be the last call. Let it be preserved in this essay.

Some Features of PostgreSQL Logical Replication

Mon, 12 Aug 2024 00:00:00 +0000

I’ve already written a fairly detailed article about logical replication before, so I won’t repeat the basics here. However, some knowledge points inevitably get missed. Recently I’ve discovered some interesting logical replication features.

replica identity and old/new values
#

replica identity is used to identify a row during logical replication. The above statement is certainly correct, but it doesn’t explain the changes in old and new data.

DEFAULT Records the old values of the columns of the primary key, if any. This is the default for non-system tables. USING INDEX index_name Records the old values of the columns covered by the named index, that must be unique, not partial, not deferrable, and include only columns marked NOT NULL. If this index is dropped, the behavior is the same as NOTHING. FULL Records the old values of all columns in the row. NOTHING Records no information about the old row. This is the default for system tables.

The PG official documentation only explains the old value situation for replica identity — for example, it doesn’t even mention that NOTHING won’t replicate update/delete. This shows the importance of old values.

Creating a replication link:

select pg_create_logical_replication_slot('pubtestlzl2','test_decoding');
pg_recvlogical -d lzldb --slot=pubtestlzl2 --start -f recv.sql &

Normal test_decoding replication link simulation:

--replica identity defaults to d: uses primary key when available; without primary key, defaults to nothing, unable to replicate update and delete
M=> create table lzltest(a bigint primary key,b varchar(100),c varchar(100));
CREATE TABLE
M=> insert into lzltest values(1,'bbbbbb','ccccccccc');
INSERT 0 1
M=> update lzltest set b='b';
UPDATE 1

recvlogical output:

table public.lzltest: INSERT: a[bigint]:1 b[character varying]:'bbbbbb' c[character varying]:'ccccccccc'
table public.lzltest: UPDATE: a[bigint]:1 b[character varying]:'b' c[character varying]:'ccccccccc'

With replica identity as default, updating a non-primary-key field — all fields have only new values.

M=> update lzltest set a='111';
UPDATE 1
table public.lzltest: UPDATE: old-key: a[bigint]:1 new-tuple: a[bigint]:111 b[character varying]:'bb' c[character varying]:'ccccccccc'

With replica identity as default, updating the primary key — the identity column’s old and new values are decoded; other fields only have new values.

M=> alter table lzltest replica identity full;
ALTER TABLE

M=> update lzltest set b='b';
UPDATE 1
table public.lzltest: UPDATE: old-key: a[bigint]:2 b[character varying]:'b' c[character varying]:'ccccccccc' new-tuple: a[bigint]:2 b[character varying]:'b' c[character varying]:'ccccccccc'

With replica identity set to full, both old and new values for the entire row are preserved.

Whether in default (primary key) or full mode, all column information is recorded. The difference lies in whether old data is present. In default mode:

insert: inherently new data, so naturally no old values — all column new values are recorded.
update: records new values for all columns; only the identity column has old values (if the identity column was updated).
delete: inherently old data, but not all columns are necessarily recorded. The same rule applies: only the identity column has old values — only the identity column is recorded.

Summary: When replica identity is default, regardless of the operation (INSERT, UPDATE, DELETE), as long as it’s old data, only the identity column is recorded; as long as it’s new data, all columns are recorded.

When changing from default to full, the decoded log volume difference isn’t particularly large, because new data always includes all columns. (Excluding scenarios that are entirely deletes) the log volume decoded under full is less than twice that of default.

pgoutput cannot be peeked
#

Create a replication slot using pgoutput:

select pg_create_logical_replication_slot('pubtestlzl','pgoutput');

Then try to peek or receive — both fail:

select * from pg_logical_slot_peek_changes('pubtestlzl',null,null);
pg_recvlogical -d lzldb --slot=pubtestlzl --start -f recv.sql &

pg_recvlogical: error: could not send replication command "START_REPLICATION SLOT "pubtestlzl" LOGICAL 0/0": ERROR: client sent proto_version=0 but we only support protocol 1 or higher
CONTEXT: slot "pubtestlzl", output plugin "pgoutput", in the startup callback
pg_recvlogical: disconnected; waiting 5 seconds to try again

You cannot peek or use pg_recvlogical to receive from a pgoutput replication slot. Since pgoutput is the output plugin for publish-subscribe, this plugin cannot be manually peeked or received…

Publish-Subscribe Doesn’t Have to Be PG-to-PG
#

create publication and create subscription are PG internal commands that can also be used to create links between PG databases. Third-party software can similarly use create publication and simulate subscriptions to create replication slots. This is better than directly creating replication slots because publications can manage replicated tables.

TOAST and Logical Decoding
#

TOAST columns being sent are NOT decoded! This means an entire row of data may only have part of it transmitted (when TOAST columns themselves haven’t been updated).

Normal decoding decodes all columns:

--Create a test_decoding replication slot
=> select pg_create_logical_replication_slot('logical_dest','test_decoding');
 pg_create_logical_replication_slot 
------------------------------------
 (logical_dest,349/A80040E0)
(1 row)

--Create a table with small columns
=> create table test1(a int primary key,b varchar(100),c varchar(100));
CREATE TABLE

=> select* from pg_replication_slots;
 slot_name | plugin | slot_type | datoid | database | temporary | active | active_pid | xmin | catalog_xmin | restart_lsn | confirmed_flush_lsn | wal_status | safe_wal_size 
--------------+---------------+-----------+--------+----------+-----------+--------+------------+--------+--------------+--------------+---------------------+------------+---------------
 logical_dest | test_decoding | logical | 418679 | lzldb | f | f | [null] | [null] | 872483335 | 349/A80040A8 | 349/A80040E0 | reserved | [null]
(1 row)


=> insert into test1 values (1,'qwer','qwer');
INSERT 0 1
Time: 0.915 ms
=> select * from pg_logical_slot_peek_changes('logical_dest',null,null);
 lsn | xid | data 
--------------+-----------+--------------------------------------------------------------------------------------------------
 349/A8004C78 | 872483335 | BEGIN 872483335
 349/A80103E8 | 872483335 | COMMIT 872483335
 349/A8018B30 | 872483369 | BEGIN 872483369
 349/A8018B30 | 872483369 | table public.test1: INSERT: a[integer]:1 b[character varying]:'qwer' c[character varying]:'qwer'
 349/A8018C50 | 872483369 | COMMIT 872483369
(5 rows)
--insert is decoded, containing all columns

=> update test1 set b='zxcv' where c='qwer';
UPDATE 1
Time: 4.005 ms
=> select * from pg_logical_slot_peek_changes('logical_dest',null,null);
 lsn | xid | data 
--------------+-----------+--------------------------------------------------------------------------------------------------
 349/A8004C78 | 872483335 | BEGIN 872483335
 349/A80103E8 | 872483335 | COMMIT 872483335
 349/A8018B30 | 872483369 | BEGIN 872483369
 349/A8018B30 | 872483369 | table public.test1: INSERT: a[integer]:1 b[character varying]:'qwer' c[character varying]:'qwer'
 349/A8018C50 | 872483369 | COMMIT 872483369
 349/A801D018 | 872483378 | BEGIN 872483378
 349/A801D018 | 872483378 | table public.test1: UPDATE: a[integer]:1 b[character varying]:'zxcv' c[character varying]:'qwer'
 349/A801D098 | 872483378 | COMMIT 872483378
(8 rows)
--update is decoded, containing all columns

Normally, without TOAST, decoded data includes all columns of the row.

TOAST decoding test:

--Enlarge the columns
=> alter table test1 alter column b type varchar(3000);
ALTER TABLE
Time: 8.091 ms
=> alter table test1 alter column c type varchar(3000);
ALTER TABLE
Time: 0.937 ms

--A batch random function
=> create or replace function f_random_str(length INTEGER) returns character varying
-> LANGUAGE plpgsql
-> AS $$
-> DECLARE
-> result varchar(3000);
-> BEGIN
-> SELECT array_to_string(ARRAY(SELECT chr((65 + round(random() * 25)) :: integer)
-> FROM generate_series(1,length)), '') INTO result;
-> return result;
-> END
-> $$;
CREATE FUNCTION

--Insert data
=> insert into test1 values (2,f_random_str(2000),f_random_str(2000));
INSERT 0 1

--Check for TOAST
=> SELECT 
-> n.nspname as schema, 
-> s.oid::regclass as relname, 
-> s.reltoastrelid::regclass as toast_name, 
-> pg_relation_size(s.reltoastrelid) AS toast_size 
-> FROM 
-> pg_class s join pg_namespace n 
-> on s.relnamespace = n.oid 
-> WHERE 
-> relkind = 'r' 
-> AND reltoastrelid <> 0 
-> AND n.nspname = 'public' 
-> ORDER BY 
-> 3 DESC;
 schema | relname | toast_name | toast_size 
--------+---------+--------------------------+------------
 public | test1 | pg_toast.pg_toast_418714 | 8192
(1 row)

--Update via primary key, updating a TOAST column
=> update test1 set b='zxcv' where a=2;
UPDATE 1

=> select * from pg_logical_slot_peek_changes('logical_dest',null,null);
 lsn | xid | 
--------------+-----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
...
 349/A851FD90 | 872483420 | BEGIN 872483420
 349/A85216E0 | 872483420 | table public.test1: INSERT: a[integer]:2 b[character varying]:'GIORCXQQWDBGTUNDZXAWMPYOUEGTECWTVQGDQGSPMEPJNPUQIFMESLRASBZWGONETRENDCHLDWVTDWJLTGRYUMFDOWHLEYLUTECPOVCYXFIATLKVEQTHSC'
 349/A85218A0 | 872483420 | COMMIT 872483420
 349/A8525CA8 | 872483429 | BEGIN 872483429
 349/A8525D50 | 872483429 | table public.test1: UPDATE: a[integer]:2 b[character varying]:'zxcv' c[character varying]:unchanged-toast-datum
 349/A8525DE0 | 872483429 | COMMIT 872483429

Column c, which has TOAST and was not involved in the update, has no decoded data — it directly outputs toast datum unchanged: unchanged-toast-datum.

Testing with wal2json:

=> select pg_create_logical_replication_slot('logical_json','wal2json');
 pg_create_logical_replication_slot 
------------------------------------
 (logical_json,349/A87CAB58)
(1 row)

=> update test1 set b='zxcv' where a=2;
UPDATE 1
=> \pset format wrapped
Output format is wrapped.
=> \pset columns 200
Target width is 200.
=> select * from pg_logical_slot_peek_changes('logical_json',null,null);
-[ RECORD 1 ]------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
lsn | 349/A87CACF8
xid | 872483495
data | {"change":[{"kind":"update","schema":"public","table":"test1","columnnames":["a","b"],"columntypes":["integer","character varying(3000)"],"columnvalues":[2,"zxcv"],"oldkeys":{"keynames":["a"],.
 |."keytypes":["integer"],"keyvalues":[2]}}]}

=> update test1 set b='zxcv' where a=1;
UPDATE 1
Time: 1.391 ms
=> select * from pg_logical_slot_peek_changes('logical_json',null,null);
-[ RECORD 1 ]------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
lsn | 349/A87CACF8
xid | 872483495
data | {"change":[{"kind":"update","schema":"public","table":"test1","columnnames":["a","b"],"columntypes":["integer","character varying(3000)"],"columnvalues":[2,"zxcv"],"oldkeys":{"keynames":["a"],.
 |."keytypes":["integer"],"keyvalues":[2]}}]}
-[ RECORD 2 ]------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
lsn | 349/A8CCA0D8
xid | 872483509
data | {"change":[{"kind":"update","schema":"public","table":"test1","columnnames":["a","b","c"],"columntypes":["integer","character varying(3000)","character varying(3000)"],"columnvalues":[1,"zxcv".
 |.,"qwer"],"oldkeys":{"keynames":["a"],"keytypes":["integer"],"keyvalues":[1]}}]}
 --When updating, column c data is not decoded

wal2json shows the same behavior.

MySQL’s binlog_row_image parameter can adjust whether binlog records large fields:

full (Log all columns)
minimal (Log only changed columns, and columns needed to identify rows)
noblob (Log all columns, except for unneeded BLOB and TEXT columns)

PG has absolutely no such control — by default, TOAST columns are not decoded, and there are no other options to configure~

Chatting About American TV Shows — June 2023

Thu, 01 Jun 2023 00:00:00 +0000

I just finished watching the Yellowstone series and decided to write a bit about the American shows I’ve watched recently — eleven in total. Here’s a quick review of each.

Yellowstone
#

Yellowstone is already at its fifth season, and it looks like they’ll keep going. When I first started watching, I genuinely got hooked — a beautiful, grand series with stunning cinematography and gorgeous scenery. Plus, you get to see how real American ranchers herd cattle — actual ranchers really do have that old-money landowner vibe…

Season one’s plot holds up fine — the dynamics between the Dutton family, the Native Americans, the state government, and the developers work well, and you can casually enjoy watching cowboys herd cattle along the way. But the plot in later seasons… is unexpectedly bad. Downright incomprehensible. It lowers the bar for screenwriting.

Zooming into the show’s core: why do so many people love this series? Because Yellowstone doesn’t just depict authentic cowboy life (they even filmed some genuine ranch cowboy life later on) — it also reflects the harsh reality that old ranches can barely survive under modern societal development. And cowboy culture and private land are the very heart of American identity. It’s not just the Dutton family stubbornly trying to preserve the ranching way of life — it almost feels like a clash between urban American development and native cultural preservation.

I can responsibly say: the plot definitely gets worse with each season — so bad that the main storyline becomes unwatchable. But if they release more seasons, this show will still be my top priority over everything else.

Personal rating: ⭐️⭐️⭐️⭐️⭐️

Recommended: ⭐️⭐️⭐️

1923
#

A Yellowstone prequel series. Maybe because Yellowstone is so famous, this prequel 1923 ended up with a bit too many stars — it left a bad impression from the start. Perhaps the creators thought Yellowstone’s plot wasn’t good enough, and that a show purely about cowboys would be hard to craft a compelling script for, so they added two subplots to 1923. But adding subplots created another problem: the show doesn’t feel enough like Yellowstone. Constant cutting between storylines — no “slow-paced” Yellowstone vibe.

The Native American girl’s storyline seems completely disconnected from the main plot — no idea when it’ll tie in. But this Native girl subplot is actually pretty good. Native lands were stolen, and their children were sent to boarding schools to be forcibly indoctrinated with white Christian beliefs. This subplot genuinely carries the Yellowstone spirit. The Native characters are cold-blooded killers too — none of that “bullet in the body but still politicking” dissonance. The narrative flows smoothly without dragging; this subplot is quite watchable.

As for the Africa subplot… while they do capture some scenery, it’s just not as good as the Dutton ranch — doesn’t have that feeling. And once they leave Africa, it starts dragging, heavily focusing on a grand romance set against the era’s backdrop — but what does that have to do with Yellowstone? And this storyline waited an entire season without converging into the main plot… An entire season of setup for one character, framed as “the Dutton ranch’s hope rests on him” — the stakes are too high, and the subplot itself isn’t that compelling. Season two is highly likely to be a massive flop.

The early part of 1923 still had some ranch-versus-the-tide-of-history flavor. Later it’s pure padding — they don’t even film cattle herding anymore. Completely devoid of interest. Can’t even muster a decent fight. Kind of bad. Only eight episodes in the whole season, and the plot starts falling apart halfway through — didn’t learn anything from Yellowstone except how to botch the ending.

You can tell this show wanted to inherit Yellowstone but also try something new — depicting that era’s America and Europe (even colonial Africa) — but ended up being a mess of everything and nothing. If you want to revisit that era, I recommend Boardwalk Empire, which is set around the same time (Prohibition era) and has far more period atmosphere than this show.

Personal rating: ⭐️⭐️⭐️

Recommended: ⭐️⭐️

1883
#

1883 — a grand, tragic Western epic. A Yellowstone series entry, the prequel to the prequel. It feels like watching an epic saga, leaving you wanting more. It’s no longer just a simple TV show — the cinematography even has literary and artistic qualities, while also carrying a slice of American pioneering history. The U.S. had just emerged from the Civil War, everything was waiting to be rebuilt…

I personally really enjoy shows like Yellowstone — the filming style suits my taste. But the main series plot is aggressively terrible; I’d rather just watch them ride horses on the ranch and skip the main storyline entirely. 1883 fills that gap perfectly — not too much complex plot (but not too little either), just right. Look at the valley, look at the horses, add some epic BGM, and the immersion is strong.

The entire 1883 series doesn’t actually have much plot, but it tells a very complete story. America had just ended its Civil War, in an era of lawlessness — cowboys, bandits, sheriffs, European immigrants, Native Americans… There’s some classic cowboy shootout action, but the focus is more on cowboy life and immigrants’ yearning for freedom. Yet the road to freedom is full of hardship: horse thieves, Native tribes, rattlesnakes, tornadoes, and this unforgiving land. A deeply profound show. Other than the female lead’s runny nose being a minus, there’s nothing to criticize. The plot is that rare combination of complete and perfectly proportioned. Very, very highly recommended.

Here’s a favorite line describing cowboys:

Personal rating: ⭐️⭐️⭐️⭐️⭐️

Recommended: ⭐️⭐️⭐️⭐️⭐️

Tulsa King
#

A pure thrill ride. Starring 70-something Sylvester Stallone as an old-school mobster who’s been locked up for decades, now reasserting order over a small city’s underworld. “It’s not that I can’t adapt — it’s that people today have messed-up rules.” Us old-school gangsters follow a code~ The plot has no real flaws, no dragging — just pure entertainment. Not sure if they’ll keep making more.

Personal rating: ⭐️⭐️⭐️⭐️

Recommended: ⭐️⭐️⭐️⭐️⭐️

Wednesday
#

A fun watch, pretty decent. I’d never seen a gothic Lolita-style American show before, and it looks pretty good. The early parts are quite engaging and fresh. Later, when it leans into mystery, it falls off — everyone can tell who’s behind it, except Wednesday (the main character)… (A lot of American mystery shows are like this — start strong, then gradually fall apart.) If you’ve never tried the gothic Lolita style, give it a shot.

Personal rating: ⭐️⭐️⭐️⭐️

Recommended: ⭐️⭐️⭐️

The Last of Us
#

Adapted from the video game of the same name — which I somehow never played! Precisely because I hadn’t played it, I could watch the show with a calm mind. Starring the hugely popular Lyanna Mormont (Bella Ramsey) and Oberyn Martell (Pedro Pascal) from Game of Thrones — both deliver smooth, natural performances. It’s a post-apocalyptic zombie-type show, but the zombies aren’t from a virus — they’re from a fungal infection. The zombies’ brains are full of fungus. One memorable scene: Bella’s character cuts open the head of a zombie stuck between rocks, and the fungus inside spills out — still alive. Maybe because of the fungus element, it’s more satisfying than the average zombie show. The visuals are great — not dark and murky, and not overly disgusting. A complete, well-told story with excellent cinematography. There’s one segment near the end that personally left me with some psychological discomfort, but overall the plot absolutely holds up. Several smaller storylines are beautifully told. Very good overall, highly recommended.

Personal rating: ⭐️⭐️⭐️⭐️

Recommended: ⭐️⭐️⭐️⭐️⭐️

Boardwalk Empire
#

A series spanning four seasons, now complete. Set in 1920s America, right after Prohibition was enacted. Women stood outside bars calling for their rights; politicians publicly supported women while privately running bootlegging operations; gangsters stepped out of cars in trench coats, Thompson submachine guns blazing… Boardwalk Empire is about the gangster empire of Atlantic City (just below New York), built on bootlegging into a wealth rivaling nations. I imagine many have seen Once Upon a Time in America — you can roughly think of this show as its TV series counterpart. This one is hard to summarize — let’s go season by season.

Season one is god-tier. Plenty of risqué scenes, and the plot isn’t just smooth — it’s miraculous. Women, black communities, bootlegging, jazz, gang wars, WWI veterans… Gangsters have essentially seized control of the city — even the newspapers don’t care what the mayor says.

Season two is a direct continuation of season one — also excellent.

Season three introduces problems. It doesn’t feel like a continuation of the first two seasons (though some plot threads connect) — it could almost stand alone. Is the plot bad? Yes, it’s disconnected. But is it terrible? Taken on its own, it’s not flawed — it’s even somewhat entertaining. This season has many brilliant segments: Half-Face taking on ten men alone, the jaw-dropping plotline of the formidable madam, extended solo blues performances by black characters — all superb!

Season four is full of issues. I thought my favorite character, dormant for three seasons, would finally take center stage and do something meaningful — instead, he was hastily written off. Dear writers, if that’s how it was going to be, could you not have put him on the poster? Made it seem like something big was coming — got my hopes up for nothing… Season four’s protagonist has risen too high, making it hard to drive the plot (you could already feel this in season three). The only highlight of season four is the protagonist’s childhood flashbacks — a perfect closure to his arc.

Many characters’ later arcs are unsatisfying, but many characters’ mid-series arcs are just too brilliant… Although this show isn’t hugely popular, it did win awards, and you can see many scenes being referenced by later, higher-profile American shows. For example, Gus Fring’s arc in Breaking Bad borrows from Half-Face; King Tommen’s suicide in Game of Thrones “completely” borrows from the butler’s suicide…

I really love this show — it immerses you in the glamorous cities of that era, the decadent urban life, the jazz of underground speakeasies, the gangsters… A narrative that holds nothing back (I mean that about everything). The series as a whole is excellent, rich with period atmosphere.

Personal rating: ⭐️⭐️⭐️⭐️⭐️

Recommended: ⭐️⭐️⭐️⭐️

Band of Brothers
#

I’m sure many have heard of this show’s reputation. Yes — I somehow hadn’t seen it. My elementary-school-level writing ability and limited education prevent me from offering any meaningful critique. Only one word can describe it: divine. I’ll find a chance to watch it again~

Personal rating: ⭐️⭐️⭐️⭐️⭐️

Recommended: ⭐️⭐️⭐️⭐️⭐️

The Pacific
#

The Pacific was made shortly before Band of Brothers. It’s actually a very good show, but then that monster came along, and this one’s reputation never reached the same heights. Band of Brothers covers the European theater of WWII; this show covers the Pacific theater. Strangely, the two shows mirror their respective theaters — the European theater is far better known, and the shows follow suit… Even within the show, at the same dinner table, a European theater soldier shows off a captured Nazi banner while the Pacific theater soldier has nothing to show — a touch of melancholy.

Though not as famous, this is a very, very highly recommended show.

Personal rating: ⭐️⭐️⭐️⭐️⭐️

Recommended: ⭐️⭐️⭐️⭐️⭐️

The Mandalorian
#

The Mandalorian is already at its third season. Also starring Pedro Pascal, also a dad-with-kid storyline… The first two seasons were quite good and fairly popular. This third season? Not so much. The Mandalorian should probably remain a ronin-like figure driving the plot forward — a whole group of Mandalorians building a homeland just doesn’t feel right. The protagonist’s identity even gets a bit diluted. (Run, man — take the kid and adventure across the galaxy — isn’t that better?)

My appreciation for this show is premised on liking the Star Wars universe. In China, Star Wars fans are genuinely rare. If you’re not into it, you probably won’t get through it — feel free to skip.

Personal rating: ⭐️⭐️⭐️⭐️⭐️

Recommended: ⭐️

The White Tower (Shiroi Kyoto)
#

This one is a Japanese drama. I want to end with it, because it truly is exceptional — near perfect. Though it’s somewhat old, it never feels boring while watching. Many ideas are surprisingly forward-thinking, the plot rises and falls dramatically, good and evil are never absolute, and several female characters are beautifully drawn. You’ll see some classic love triangles and plot twists, and revisiting them is still quite rewarding. Professor Zaizen’s final act brings the entire series to a perfect close. Japanese drama — number one!

Personal rating: ⭐️⭐️⭐️⭐️⭐️

Recommended: ⭐️⭐️⭐️⭐️⭐️

Closing
#

All of these are worth watching, and many are masterpieces. Some shows I couldn’t find subtitled versions for, so I watched them raw — like the Yellowstone prequel 1883. Since the dialogue wasn’t overly complex, I managed to get through it (the narration is quite sophisticated)… Marking my first raw viewing.

These are basically all the shows I’ve watched in the last half year or so, so I’m bundling them together. There are many other brilliant shows from earlier that left a deep impression — I’ll save that for another time when I’m in the mood~

Hoping to find more good shows in the second half of the year.

杂项 on Last DBA

A DBA's Perspective on the 0526 Approved Database List

TL;DR #

The Latest List #

The Floodgates Open #

Impact #

Bittersweet Reflections #

Reference #

My Blog is Live

It’s Live! #

Highlights #

Reflections on Going Live #

What It Cost #

Finally #

Reference #

My 2025 Year-End Summary

As a DBA #

READING #

Blog and WeChat Official Account #

Final Thoughts #

My 2024 Year-End Summary

As a DBA #

READING #

Blog and WeChat Official Account #

Final Thoughts #

Getting Started with pg_rewind

What is pg_rewind? #

Using pg_rewind #

Common Issues #

pg_rewind Error 1 #

pg_rewind Error 2 #

pg_rewind Error 3 #

References #

How I Got Into Wuhan University's Part-Time Master's Program

Why Did I Want to Pursue a Part-Time Master’s? #

Differences Between Full-Time and Part-Time Graduate Programs #

Study Mode #

Exam Scope #

Research Direction #

Social Recognition #

How to Choose a Major #

How to Choose a School #

How Does the Exam Work? #

Exam Content #

Early Interview #

How to Register #

Should You Sign Up for a Training Course? #

What Did the Training Course Give Me? #

The Preliminary Exam #

How to Prepare for the Preliminary Exam — Study Plan #

Recommended Study Materials #

English Study #

Vocabulary #

Reading #

Writing #

Math and Logic Study #

Math questions: #

Logic questions: #

Chinese Writing Study #

Exam Time Strategy #

The Re-examination #

Basic Information About the Re-examination #

Post-Adjustment (Tiaoji) #

How to Prepare for the Re-examination? #

The End #

OGG Oracle-to-PostgreSQL Sync — Hands-On Steps

1. Install OGG Software on Source and Target #

2. Set Database to Archive Mode #

3. Enable Force Logging and Minimum Supplemental Logging #

4. Set enable_goldengate_replication Parameter #

5. Create OGG User, Tablespace, and Grant Privileges #

6. Enable Table-Level Supplemental Logging #

7. Configure Manager Process #

8. Configure Extract Process #

9. Configure Pump Process #

10. Start Extract Process #

11. Configure Target OGG Software #

12. Create Database and Table on Target #

13. Configure Target Manager Process and Start #

14. Target PostgreSQL Parameter Adjustment #

TL;DR
#

The Latest List
#

The Floodgates Open
#

Impact
#

Bittersweet Reflections
#

Reference
#

It’s Live!
#

Highlights
#

Reflections on Going Live
#

What It Cost
#

Finally
#

Reference
#

As a DBA
#

READING
#

Blog and WeChat Official Account
#

Final Thoughts
#

As a DBA
#

READING
#

Blog and WeChat Official Account
#

Final Thoughts
#

What is pg_rewind?
#

Using pg_rewind
#

Common Issues
#

pg_rewind Error 1
#

pg_rewind Error 2
#

pg_rewind Error 3
#

References
#

Why Did I Want to Pursue a Part-Time Master’s?
#

Differences Between Full-Time and Part-Time Graduate Programs
#

Study Mode
#

Exam Scope
#

Research Direction
#

Social Recognition
#

How to Choose a Major
#

How to Choose a School
#

How Does the Exam Work?
#

Exam Content
#

Early Interview
#

How to Register
#

Should You Sign Up for a Training Course?
#

What Did the Training Course Give Me?
#

The Preliminary Exam
#

How to Prepare for the Preliminary Exam — Study Plan
#

Recommended Study Materials
#

English Study
#

Vocabulary
#

Reading
#

Writing
#

Math and Logic Study
#

Math questions:
#

Logic questions:
#

Chinese Writing Study
#

Exam Time Strategy
#

The Re-examination
#

Basic Information About the Re-examination
#

Post-Adjustment (Tiaoji)
#

How to Prepare for the Re-examination?
#

The End
#

1. Install OGG Software on Source and Target
#

2. Set Database to Archive Mode
#

3. Enable Force Logging and Minimum Supplemental Logging
#

4. Set enable_goldengate_replication Parameter
#

5. Create OGG User, Tablespace, and Grant Privileges
#

6. Enable Table-Level Supplemental Logging
#

7. Configure Manager Process
#

8. Configure Extract Process
#

9. Configure Pump Process
#

10. Start Extract Process
#

11. Configure Target OGG Software
#

12. Create Database and Table on Target
#

13. Configure Target Manager Process and Start
#

14. Target PostgreSQL Parameter Adjustment
#

15. Data Source Configuration (odbc.ini)
#

16. Connection Test
#

17. Configure and Start Replicat Process on Target
#

18. Test Verification
#

1. Create Database and Table on Source
#

2. Create Database and Table on Target
#

3. Extract and Install OGG for PostgreSQL
#

4. Configure PG User Environment Variables
#