vvvv 3d5bcb1e40 less ydb deps or disable for OSS for now 1 week ago
..
interface 2661be00f3 Moved yql/minikql YQL-19206 1 week ago
postgresql d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
test 3d5bcb1e40 less ydb deps or disable for OSS for now 1 week ago
ut d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
arena_ctx.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
arena_ctx.h d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
arrow.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
arrow.h d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
arrow_impl.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
arrow_impl.h d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
cflags.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
comp_factory.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
config.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
conversion.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
copy_src.py d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
copy_src.sh d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
cost_mocks.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
ctors.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
ctors.h d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
generate_kernels.py d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
memory_context.h d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
parser.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_aggs.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_aggs.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_aggs.slow.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_aggs_register.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_bc.0.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_bc.1.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_bc.10.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_bc.11.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_bc.12.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_bc.13.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_bc.14.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_bc.15.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_bc.16.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_bc.17.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_bc.18.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_bc.19.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_bc.2.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_bc.3.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_bc.4.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_bc.5.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_bc.6.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_bc.7.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_bc.8.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_bc.9.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_bc.all.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_catalog_consts.h d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_compat.h d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernel_sources.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.0.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.0.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.1.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.1.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.10.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.10.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.11.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.11.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.12.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.12.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.13.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.13.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.14.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.14.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.15.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.15.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.16.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.16.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.17.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.17.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.18.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.18.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.19.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.19.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.2.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.2.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.3.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.3.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.4.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.4.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.5.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.5.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.6.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.6.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.7.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.7.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.8.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.8.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.9.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.9.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.slow.0.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.slow.1.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.slow.10.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.slow.11.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.slow.12.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.slow.13.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.slow.14.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.slow.15.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.slow.16.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.slow.17.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.slow.18.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.slow.19.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.slow.2.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.slow.3.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.slow.4.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.slow.5.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.slow.6.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.slow.7.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.slow.8.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels.slow.9.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels_fwd.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels_register.0.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels_register.1.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels_register.10.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels_register.11.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels_register.12.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels_register.13.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels_register.14.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels_register.15.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels_register.16.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels_register.17.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels_register.18.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels_register.19.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels_register.2.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels_register.3.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels_register.4.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels_register.5.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels_register.6.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels_register.7.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels_register.8.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels_register.9.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_kernels_register.all.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_proc_policies.0.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_proc_policies.1.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_proc_policies.10.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_proc_policies.11.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_proc_policies.12.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_proc_policies.13.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_proc_policies.14.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_proc_policies.15.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_proc_policies.16.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_proc_policies.17.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_proc_policies.18.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_proc_policies.19.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_proc_policies.2.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_proc_policies.3.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_proc_policies.4.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_proc_policies.5.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_proc_policies.6.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_proc_policies.7.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_proc_policies.8.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_proc_policies.9.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_proc_policies.all.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_sources.inc d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
pg_utils_wrappers.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
read_table.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
readme.md d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
recovery.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
source.patch d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
source16.patch d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
superuser.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
syscache.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
thread_inits.c d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
thread_inits.h d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
type_cache.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
type_cache.h d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
utils.cpp d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
utils.h d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
vars.txt d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
verify.sh d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
win32_pg_compat.h d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago
ya.make d4c258e943 Moved other yql/essentials libs YQL-19206 1 week ago

readme.md

Directory postgresql/ contains PostgreSQL sources that were automatically patched to make all global variables to become thread-local ones. This makes it possible to use pg functions in multi-thread environment.

Also, there are some additional commits to postgresql/ (mostly backports from newer PostgreSQL versions or renames of PostgreSQL C functions when corresponding function is reimplemented outside of postgresql/ directory/)

Base version of PostgreSQL can be seen in copy_src.sh script (VERSION variable).

copy_src.sh script downloads, configures and builds static postgresql.a library from original PostgreSQL sources with minor patches (source.patch). After that, it finds all global r/w symbols by analyzing output of "objdump postgresql.a" command. Then it copies all necessary *.c and *.h files to postgresql/ directory, automatically changing all static variables to thread-local ones.

How to upgrade to newer PostgreSQL:

  1. Make sure you have folloing packages installed (assuming you have Ubuntu Linux):

libicu-dev icu-devtools pkg-config liblz4-dev libreadline-dev libssl-dev libxml2-dev libossp-uuid-dev

  1. On clean repository perform following command:

./copy_src.sh && git diff -R postgresql > local_changes.patch

In file local_changes.patch you will get all changes applied to postgresql/ directory which are not part of automated patching by copy_src.sh

  1. Cleanup repositry:

git checkout .

  1. Bump PostgreSQL version in copy_src.sh (it is recommended not to do big jumps here)

  2. Run ./copy_src.sh

  3. Assuming compilation and automatic patching were successful, apply local changes collected on step 2:

patch -p6 < local_changes.patch

Resolve possible conflicts. Usually, conflicts arise due to some already backported changes

  1. Update pg_catalog data

(cd ../../tools/pg_catalog_dump/ && yag make --build=relwithdebinfo && ./pg_catalog_dump > dump.json)

  1. Regenerate Arrow postgresql kernels

./generate_kernels.py

  1. Make sure that resulting pg_wrapper library compiles and passes minimal tests

yag make --build=relwithdebinfo -tA -C ut -C test -C ../../sql/pg/ut

  1. Verify that all global variables in PostgreSQL-originated sources are accounted for

./verify.sh

Should output OK

  1. Submit PR (do not forget to git add all new files created in postgresql/ direcotry - they sshould also present in pg_sources.inc)