Tree:
a261a76f57
24-1-mkql-runtime
add-upcoming-event-1
alexv-smirnov-patch-1
alexv-smirnov-patch-2
alexv-smirnov-patch-3
alexv-smirnov-patch-4
blinkov-patch-1
blinkov-patch-2
bs/vdisk/split-synclog-processing
cmake-actions-1
cmake2
cmake3
cmakebuild
dev-dirty_fix_clang14
dev-jinja-html
dev-jinja-html2
dev-night
dev-nightly
dev-remove_clang14_from_pr
dev-sammy-optimization
dev-stable-24-3
dev-test-policy
dev-warmup
dev-whole-analytics
dev-whole-analytics-sammy
dev_dirty_fix_clang14
docker-connect-doc-up
docs-3-nodes-mirror-3-dc
docs-371-tmp
docs-633-review-follow-up
docs-add-coscup
docs-add-database-node-synonym
docs-add-gitignore
docs-add_translation_for_terraform_ydb_doc
docs-ansible-add-extra-note
docs-ansible-installation-options
docs-ansible-mistype-restart
docs-ansible-ydb-archive
docs-bugfix
docs-build-fixes
docs-change-storage-bucket
docs-cli-recipes
docs-cli-restore-import-data
docs-codowners-root-md
docs-columnar-pk
docs-concepts-cluster
docs-contents-snippet
docs-create-table-ident
docs-dark-theme
docs-data-at-rest-improvement
docs-default-sql-version
docs-dev-getting-started-mistype
docs-distconf-tmp
docs-docker-old-version-note
docs-dont-link-to-archived-go-sdk
docs-drop-old-public-talks
docs-drop-old-public-talks-1
docs-drop-unused-files
docs-edit-ru-ansible-articles
docs-enable-feature-codegen
docs-example-app-en-fixes
docs-fix-ansible-repo-name
docs-fix-column-table-redirect
docs-fix-concat-md
docs-fix-internal-build
docs-fix-internal-build-1
docs-fix-oauth2-link
docs-fix-pg-create-include
docs-fix-rendering
docs-fix-ru-text-in-en
docs-fix-test-commit-ui
docs-fq-postgresql-fix-link
docs-grafana-columnar
docs-gtm
docs-hotfix-config-examples
docs-hotfix-public-talks-redirect
docs-increase-disabled-nodes-ratio-limit
docs-indiafoss-2024
docs-insert-into-hotfix
docs-kikimr-20945-pin-configs
docs-kubecon-hongkong
docs-manual-deployment-refactoring
docs-md-fix-cast-examples
docs-more-internal-fixes
docs-mute
docs-old-docker
docs-pause-autodeploy
docs-pg-redirect-hotfix
docs-postgresql-docker-connect-minor-fix
docs-preview-in-pr
docs-quickstart-add-mkdir
docs-quickstart-docker-arm
docs-refactor-build-sh
docs-refactoring-ydbdocs-633
docs-release-workflow-add-paths
docs-remove-setup-playbook-mention
docs-rm-mirror-3dc-3-nodes
docs-ru-glossary-terminology-fixes
docs-ru-index-yaml
docs-security-encryption
docs-simplify-sidebar
docs-single-page
docs-stackonf-fix-url
docs-test-codeowners
docs-test-if-oss
docs-tls-mistype
docs-tmp-alter-table
docs-topic-workload-fix-option-name
docs-topology-md-improvements
docs-translate-9019
docs-translate-connections-string
docs-translate-tracing
docs-translation-debts
docs-trigger-ci
docs-try-lunr
docs-unpause-deploy
docs-unrestrict-revision-access
docs-update-decimal-note
docs-update-diplodoc-actions
docs-update-favicon
docs-update-gitignore
docs-update-license-year
docs-update-query-limits
docs-update-readme
docs-update-redirects
docs-update-workflow-type
docs-use-ansible-collections-playbook
docs-version-builtin
docs-workflow-on-create
docs-ya-make-yql
docs-ydb-go-sdk-prometheus
docs-ydb-var
docs-ydbdocs-119-glossary-revamp
docs-ydbdocs-191-translate-fq
docs-ydbdocs-519-translate-config-related
docs-ydbdocs-596
docs-ydbdocs-633-continue
docs-ydbdocs-633-continue-2
docs-ydbdocs-633-continue-3
docs-ydbdocs-633-continue-4
docs-ydbdocs-633-refactoring
docs-ydbdocs-653-restore-minikube-quickstart
docs-ydbdocs-668-add-ansible-logging
docs-ydbdocs-668-playbook-name
docs-ydbdocs-690-ansible-restart
docs-ydbdocs-691-introduce-recipes-section
docs-ydbdocs-692-update-config
docs-ydbdocs-696-fix-yql-build
docs-ydbdocs-760-analytics-faq
docs-ydbdocs-804-ansible-fq-connector
docs-ydbdocs-909
docs-ydbdocs-917
docs-ydbops-9957-updating-executables-ansible
docs-yfm-link
docs-yfm-warnings
docs-yfmlint
docs-yfmlint-2
docs-yfmlint-3
docstools-update-build-workflow
embedded-ui-refresh-v4.31.0
embedded-ui-refresh-v4.31.1
embedded-ui-v4.33.0
embedded-ui-v5.5.0
embedded-ui-v6.10.2
embedded-ui-v6.16.0
embedded-ui-v6.17.2
embedded-ui-v6.28.0
embedded-ui-v6.4.0
embedded-ui-v6.8.0
feature/KIKIMR-20843/fix-reserve
feature/backups/pre-6
feature/integrations_and_ide
fix-header-ident-bloom
license-year-2023
main
mergelibs-240308-1527
mergelibs-240312-1040
mergelibs-240313-1032
mergelibs-240313-1536
mergelibs-240318-0944
mergelibs-240325-1123
mergelibs-240326-1411
mergelibs-240326-1608
mergelibs-240327-1029
mergelibs-240403-0851
mergelibs-240404-1137
mergelibs-240408-0819
mergelibs-240416-0910
mergelibs-240417-0847
mergelibs-240423-0906
mergelibs-240425-1630
mergelibs-240506-1255
mergelibs-240507-1023
mergelibs-240508-0817
mergelibs-240510-1206
mergelibs-240514-0848
mergelibs-240603-0842
mergelibs-240606-1312
mergelibs-240610-0905
mergelibs-240611-1110
mergelibs-240618-0830
mergelibs-240621-1318
mergelibs-240626-2018
mergelibs-240708-1553
mergelibs-240719-0942
mergelibs-240730-1922
mergelibs-240803-2327
mergelibs-240807-0824
mergelibs-240807-1211
mergelibs-240807-1311
mergelibs-240814-1820
mergelibs-240814-2205
mergelibs-240815-1025
mergelibs-240816-1413
mergelibs-240819-0942
mergelibs-240819-1422
mergelibs-240819-2038
mergelibs-240820-1626
mergelibs-240820-2236
mergelibs-240821-1409
mergelibs-240823-1155
mergelibs-240830-1442
mergelibs-240903-1104
mergelibs-240905-0836
mergelibs-240906-1233
mergelibs-240911-0600
mergelibs-240913-0843
mergelibs-240916-1031
mergelibs-240918-0758
mergelibs-240924-0826
mergelibs-240926-1443
mergelibs-240930-1059
mergelibs-241002-1139
mergelibs-241002-1721
mergelibs-241003-0910
mergelibs-241003-1145
mergelibs-241003-1407
mergelibs-241004-1110
mergelibs-241004-1426
mergelibs-241007-1548
mergelibs-241008-2121
mergelibs-241009-1030
mergelibs-241011-1107
mergelibs-241015-1526
mergelibs-241016-1210
mergelibs-241019-1758
mergelibs-241021-1020
mergelibs-241028-1143
mergelibs-241028-2033
mergelibs-241031-1710
mergelibs-241101-0908
mergelibs-241102-0844
mergelibs-241106-0903
mergelibs-241108-0852
mergelibs-241115-0911
mergelibs-yql
mergelibs-yurchuk-manual
mute/naspirato
mute/naspirato-1
mute/naspirato-10
mute/naspirato-11
mute/naspirato-12
mute/naspirato-13
mute/naspirato-14
mute/naspirato-15
mute/naspirato-16
mute/naspirato-17
mute/naspirato-18
mute/naspirato-19
mute/naspirato-2
mute/naspirato-20
mute/naspirato-21
mute/naspirato-3
mute/naspirato-4
mute/naspirato-5
mute/naspirato-6
mute/naspirato-7
mute/naspirato-7-1
mute/naspirato-8
mute/naspirato-9
mute/ydb/core/kqp/workload_service/ut
mute/ydb/core/mind/hive/ut/THiveTest.DrainWithHiveRestart
mute/ydb/library/yql/providers/generic/connector/tests/datasource/clickhouse-test.py.test_select_datetime
mute/ydb/library/yql/providers/generic/connector/tests/datasource/ms_sql_server-test.py.test_select_positive
mute/ydb/tests/fq/yds-test_row_dispatcher.py.TestPqRowDispatcher.test_restart_compute_node
mute/yurchuk/kqp
mute/yurchuk/kqp2
naspirato/fix_report_html
naspirato/mute
naspirato/mute-1
naspirato/mute-2
naspirato/mute-3
naspirato/mute-4
naspirato/mute-5
naspirato/mute-6
naspirato/mute-patch-1
naspirato/patch-1
naspirato/patch-1-1
naspirato/unmute
naspirato/unmute-1
naspirato/unmute-2
naspirato/unmute-3
naspirato/update_collect_analytics
oidc-proxy-stable-2024-09
oleshiy-fix-yc-terraform
password_in_file
prestable-24-4
q-stable-2023-11-22
q-stable-2024-01-09
q-stable-2024-03-21
q-stable-2024-07-08
q-stable-ydb-24-1
q-stable-ydb-24-2
q-stable-ydb-24-3
readme-linkedin-link
readme-links-update
readme-social-update
revert-10009-dev/check_thread_in_blockdevice
revert-10337-fix_includes_3
revert-11474-mvcc-snapshot-evwrite
revert-11605-YQL-19292
revert-3499-main
revert-4919-olap-buffer
revert-4924-fix-joins
revert-5604-memory-free-fix
revert-5852-marat_nurgaleev_moi_drug
revert-6267-stats/change_as_usage
revert-6270-stats-24-1/change_as_usage
revert-6271-stats-24-2/change_as_usage
revert-6286-add-enable-impl-params-to-table-service-config
revert-7492-drop_old_json
revert-750-revert-634-kqp-computation-pattern-cache-serialized-program
revert-7994-mergelibs-240819-0942
revert-8164-ci-remove-warnings
revert-8269-patch/24-3/revert-as
revert-8383-legacy_kv_api_remove
revert-9886-libpq17-compat
revert-9998-mute/naspirato-17
rightlib
roadmap
stable-22-2
stable-22-4
stable-22-5
stable-23-1
stable-23-2
stable-23-3
stable-23-3-33-hotfix
stable-23-4
stable-23-4-11-normalizer-fix
stable-24-1
stable-24-1-10-hotfix
stable-24-1-14-analytics
stable-24-1-14-hotfix
stable-24-1-14-roaring
stable-24-1-16-analytics
stable-24-1-18-analytics
stable-24-1-8
stable-24-1-CS-patch
stable-24-1-CS-patch-2
stable-24-1-CS-patch-3
stable-24-1-CS-patch-3-att-2
stable-24-1-async-replication
stable-24-1-bump-workflows1
stable-24-1-bumpci2
stable-24-1-cbo
stable-24-1-iddqd
stable-24-1-iddqd-2
stable-24-1-iddqd-3
stable-24-2
stable-24-2-1-hotfix
stable-24-2-4-analytics
stable-24-2-4-hotfix
stable-24-2-6-hotfix
stable-24-2-7-hotfix
stable-24-2-fix-lwtrace
stable-24-3
stable-24-3-10-analytics
stable-24-3-10-hotfix
stable-24-3-11-hotfix
stable-24-3-6-logbroker
stable-24-3-7-hotfix
stable-24-3-8-analytics
stable-24-3-8-analytics-misha
stable-24-3-9-cs
stable-24-3-9-hotfix
stream-nb-1
stream-nb-1-tracing
stream-nb-1.put-lwtrack
stream-nb-2
stream-nb-2.put-lwtrack
stream-nb-24-3
testowners/naspirato
testowners/naspirato-patch-1
unmute/naspirato
unmute/naspirato-1
unmute/naspirato-2
unmute/naspirato_test
update-readme-docs-link
workflow-check-1
ydbdocs-557
yql-stable-2024-03-14
yql-stable-2024-05-07
yql-stable-2024-07-01
yql-stable-2024-09-03
yql-stable-test1
yql-temp-table
24.3.11-hotfix.7.stream-nb-24-3
24.3.11-hotfix.6.stream-nb-24-3
24.3.11-hotfix.7
24.3.13
24.3.11-hotfix.5.stream-nb-24-3.1
24.3.11-hotfix.6
24.3.11-hotfix.5.stream-nb-24-3
24.3.11-hotfix.5
24.3.11-hotfix-4
24.3.11-hotfix.4
24.3.11-hotfix.3.stream-nb-24-3
24.3.11-hotfix.3
24.3.12
24.3.11-hotfix.2
24.3.11-hotfix.1
24.3.11
24.3.10-stream-nb-24-3
CLI_2.15.0
24.3.10
24.3.9-hotfix-10.nb-stream-24-3
24.3.9-hotfix-11
CLI_2.14.0
24.3.9-hotfix-10
24.3.9-hotfix-9
24.3.9-hotfix-8
24.2.7-stream-nb-2.3
24.3.9-hotfix-7
24.3.9-hotfix-6
24.3.9-hotfix-5
24.2.7-stream-nb-2.2
24.3.9-hotfix-4
CLI_2.13.0
24.3.9-hotfix-3
24.3.9.nb-stream-24-3.3
24.3.9.nb-stream-24-3.2
24.3.9.nb-stream-24-3.1
24.3.9-hotfix-2
24.3.9.nb-stream-24-3
24.2.8
24.3.7-hotfix-7
CLI_2.12.0
24.3.9
24.3.7-hotfix-6
24.3.7-hotfix-5
24.3.8
24.2.7-stream-nb-2.1
24.3.7-hotfix-4
24.3.7-hotfix.3
24.2.7-stream-nb-2
24.3.7
24.2.7
24.3.6
24.3.5
24.3.4
24.1.18-stream-nb-1.2
24.1.18-stream-nb-1.1
24.2.6
24.2.5
24.3.3
24.1.18-stream-nb-1
CLI_2.11.0
24.2.4
24.2.3
24.2.2
24.2.1
CLI_2.10.0
24.1.16-stream-nb-1.1
24.1.18
24.1.17
24.1.16-stream-nb-1
24.1.16
24.1.15-stream-nb-1
24.1.15
24.1.12-stream-nb-1-http-header-fix
24.1.12-stream-nb-1-dns-fix
CLI_2.9.0
23.4.11
24.1.14
24.1.13
24.1.12
24.1.12-stream-nb-1
24.1.10-stream-nb-1-niam
24.1.11
23.4.6
24.1.10-stream-nb-1
24.1.10
24.1.9
24.1.8-f24b02a98f048c5a9461d20c9a15fb06eb80bb13
24.1.8-prepared-ddl
24.1.8
23.4.4
23.3.33.hotfix.2
24.1.7
23.3.33
24.1.6
24.1.5
24.1.4
24.1.3
24.1.2
24.1.1
CLI_2.8.0
23.3.25.hotfix.1
23.3.17
CLI_2.7.0
23.3.13
CLI_2.6.0
23.2.12
CLI_2.5.0
23.2.11
23.1.26.hotfix.1
CLI_2.4.0
23.1.26
CLI_2.3.0
23.1.19
CLI_2.2.0
22.5.10
CLI_2.1.1
22.4.44
22.4.43
CLI_2.1.0
22.4.31
CLI_2.0.1
22.4.30
22.4.21
22.2.47
22.2.46
22.2.40
22.2.39
22.2.38.1
22.2.38
22.2.31
22.2.22
22.2.5
robot-piglet
aaf551fb50
Intermediate changes
|
1 week ago | |
|---|---|---|
| .. | ||
| README.md | d28c55ab25 Intermediate changes | 1 year ago |
| stockpile.h | d17ef5729f Intermediate changes | 1 month ago |
| stockpile_linux.cpp | aaf551fb50 Intermediate changes | 1 week ago |
| stockpile_other.cpp | d17ef5729f Intermediate changes | 1 month ago |
| ya.make | d28c55ab25 Intermediate changes | 1 year ago |
README.md
stockpile
При приближении к лимиту памяти в memory cgroup, linux запускает механизм direct reclaim, чтобы освободить свободную память. По опыту YT, direct reclaim очень сильно замедляет работу всего процесса.
Проблема возникает не только, когда память занята анонимными страницами. 50% памяти контейнера может быть занято не dirty страницами page cache, но проблема всёравно будет проявляться. Например, если процесс активно читает файлы с диска без O_DIRECT, вся память очень быстро будет забита.
Чтобы бороться с этой проблемой, в яндексовом ядре добавлена ручка madvise(MADV_STOCKPILE).
Больше подробностей в https://st.yandex-team.ru/KERNEL-186