Logo

dev-resources.site

for different kinds of informations.

Native GLIBC instead of Linuxbrew since 2.21

Published at
10/4/2024
Categories
yugabytedb
postgres
extensions
glibc
Author
franckpachot
Main Article
https://dev.to/yugabyte/native-glibc-instead-of-linuxbrew-since-221-288h
Categories
4 categories in total
yugabytedb
open
postgres
open
extensions
open
glibc
open
Author
12 person written this
franckpachot
open
Native GLIBC instead of Linuxbrew since 2.21

TL;DR: Compiling PostgreSQL extensions is much easier in YugabyteDB versions 2.21 and above, as native OS libraries are used.

YugabyteDB initially used Linuxbrew to create a consistent, portable build environment across various Linux distributions. Linuxbrew allowed the bundling of dependencies like GCC and Glibc, ensuring controlled builds across systems, including non-Glibc-based ones. This approach simplified dependency management and testing by allowing complete control over the build environment.

However, several challenges arose with Linuxbrew, like performance overhead (relying on outdated libraries), compatibility issues (especially when compiling PostgreSQL extensions to run on yugabyteDB), and maintenance complexity.

YugabyteDB used Linuxbrew only for x86_64 builds, but aarch64 used native OS libraries. Starting with version 2.21, YugabyteDB transitioned to native libraries for all builds.

Here are the supported Operating Systems:

Operating system support | YugabyteDB Docs

Operating systems supported by YugabyteDB and YugabyteDB Anywhere.

favicon docs.yugabyte.com

Here is how to see the difference by running dd on two x86_64 images.

Version 2.19 used linuxbrew:

-bash-4.2# docker run --rm -it yugabytedb/yugabyte:2.19.3.0-b140 \
           ldd /home/yugabyte/bin/yb-tserver

        linux-vdso.so.1 (0x00007ffcbfb98000)
        librt.so.1 => /home/yugabyte/linuxbrew/lib/librt.so.1 (0x00007f5c9d7a5000)
        libresolv.so.2 => /home/yugabyte/linuxbrew/lib/libresolv.so.2 (0x00007f5c9d58d000)
        libm.so.6 => /home/yugabyte/linuxbrew/lib/libm.so.6 (0x00007f5c9d288000)
        libc.so.6 => /home/yugabyte/linuxbrew/lib/libc.so.6 (0x00007f5c9c8e7000)
        /home/yugabyte/lib/ld.so => /lib64/ld-linux-x86-64.so.2 (0x00007f5cb9a47000)
        libpthread.so.0 => /home/yugabyte/linuxbrew/lib/libpthread.so.0 (0x00007f5c9c6c8000)
        libdl.so.2 => /home/yugabyte/linuxbrew/lib/libdl.so.2 (0x00007f5c9c4c3000)
Enter fullscreen mode Exit fullscreen mode

Version 2.23 uses native libraries from (AlmaLinux 8 used for this image) :

-bash-4.2# docker run --rm -it yugabytedb/yugabyte:2.23.0.0-b710 \
           ldd /home/yugabyte/bin/yb-tserver

        linux-vdso.so.1 (0x00007ffcea370000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f4a4f00d000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f4a4edf5000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f4a4ea73000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f4a4e69d000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f4a54f53000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f4a4e47d000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f4a4e279000)
Enter fullscreen mode Exit fullscreen mode

The image is slightly smaller (you can strip the executables if the image size matters, but then you will not have all debug symbols to troubleshoot errors):

-bash-4.2# docker image ls
REPOSITORY            TAG             IMAGE ID       CREATED         SIZE
yugabytedb/yugabyte   2.23.0.0-b710   254a05710594   4 weeks ago     2.1GB
yugabytedb/yugabyte   2.19.3.0-b140   9474f15f0653   11 months ago   2.28GB
Enter fullscreen mode Exit fullscreen mode

Here is how to check the version of GLIBC (GNU libc):

-bash-4.2# docker run --rm -it yugabytedb/yugabyte:2.23.0.0-b710 \
           ldd --version

ldd (GNU libc) 2.28
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.
Enter fullscreen mode Exit fullscreen mode

YugabyteDB uses the native GLIBC libraries, so Yugabyte will validate the collations that depend on it. Most use 'libicu', but here are the collations provided by GLIBC:

yugabyte=# select collname from pg_collation 
           where collprovider = 'c';

  collname
------------
 C
 POSIX
 ucs_basic
 en_US.utf8
 en_US
Enter fullscreen mode Exit fullscreen mode

Jeremy Schneider has built an excellent tool to validate this:

GitHub logo ardentperf / glibc-unicode-sorting

Collation Changes Across Linux Versions

Methodology

GNU C Library

There are two aspects to this analysis: comparing the results of actual sorts in en_US locale, and comparing the LC_COLLATE section of the Operating System locale data files.

Comparing the results of actual sorts should catch any changes to default sorting which is not defined in the OS collation data. A simple perl script is used to generate a text file containing 91 different strings for every legal unicode character. The unix "sort" utility processes this file with the locale configured to en_US for collation. This process is repeated on each release from the past 10 years, and then the unix "diff" utility is used to compare the sorted output files and count how many characters have different positions after sorting. The results show how many individual code points have changed positions in the sorted data across different Operating System releases…

View on GitHub
yugabytedb Article's
30 articles in total
Favicon
PostgreSQL plan_cache_mode
Favicon
Index Filtering in PostgreSQL and YugabyteDB (Index Scan instead of Index Only Scan)
Favicon
Unique Index on NULL Values in SQL & NoSQL
Favicon
More details in pg_locks for YugabyteDB
Favicon
Large IntentsDB MemTable with Many Small SST Files
Favicon
Aurora DSQL: How it Compares to YugabyteDB
Favicon
Document data modeling to avoid write skew anomalies
Favicon
When to replace IN() with EXISTS() - correlated and uncorrelated subqueries
Favicon
2024.2: Faster with Shared Memory Between PostgreSQL and TServer Layers
Favicon
Aurora DSQL - Simple Inserts Workload from an AWS CloudShell
Favicon
Amazon Aurora DSQL: Which PostgreSQL Service Should I Use on AWS ?
Favicon
YugabyteDB MVCC and Updates: columns vs. JSON
Favicon
No Gap Ordered Numbering in SQL: A Unique Index to Serialize In Read Committed
Favicon
Starting a YugabyteDB lab cluster with AWS CLI
Favicon
Speeding Up Foreign Key Constraints During Migrations
Favicon
Indexing for a Scalable Serialization Isolation Level
Favicon
The Doctor's On-Call Shift example and a Normalized Relational Schema to Avoid Write Skew
Favicon
You Probably Don't Need Serializable Isolation
Favicon
A brief example of an SQL serializable transaction
Favicon
YugabyteDB as a Graph database with PuppyGraph
Favicon
Native GLIBC instead of Linuxbrew since 2.21
Favicon
pgSphere and Q3C on Distributed SQL
Favicon
IN() Index Scan in PostgreSQL 17 and YugabyteDB LSM Tree
Favicon
Frequent Re-Connections improved by Connection Manager
Favicon
Maintaining Throughput With Less Physical Connections
Favicon
YugabyteDB Connection Manager: a Database Resident Connection Pool with Shared Processes
Favicon
ERROR: index row size 3056 exceeds btree version 4 maximum 2704 for index
Favicon
Write Buffering to Reduce Raft Consensus Latency in YugabyteDB
Favicon
Asynch replication for Disaster Recovery, Read Replicas, and Change Data Capture
Favicon
Fast PITR and MVCC reads with Key-Value LSM Tree

Featured ones:

Favicon
Open
abubakersiddique761
Favicon
Open
0x3d_site
Favicon
Open
0x2e_tech
Favicon
Open
0x4c-quest
Favicon
Open
gittech