Intermediate changes

commit_hash:2835e21dc055a3e814c5f562f3a876691a772ea2

contrib/tools/python3/Lib/ctypes/macholib/README.ctypes

@@ -0,0 +1,7 @@
+Files in this directory come from Bob Ippolito's py2app.
+
+License: Any components of the py2app suite may be distributed under
+the MIT or PSF open source licenses.
+
+This is version 1.0, SVN revision 789, from 2006/01/25.
+The main repository is http://svn.red-bean.com/bob/macholib/trunk/macholib/

contrib/tools/python3/Modules/_decimal/README.txt

@@ -0,0 +1,46 @@
+
+
+About
+=====
+
+_decimal.c is a wrapper for the libmpdec library. libmpdec is a fast C
+library for correctly-rounded arbitrary precision decimal floating point
+arithmetic. It is a complete implementation of Mike Cowlishaw/IBM's
+General Decimal Arithmetic Specification.
+
+
+Build process for the module
+============================
+
+As usual, the build process for _decimal.so is driven by setup.py in the top
+level directory. setup.py autodetects the following build configurations:
+
+   1) x64         - 64-bit Python, x86_64 processor (AMD, Intel)
+
+   2) uint128     - 64-bit Python, compiler provides __uint128_t (gcc)
+
+   3) ansi64      - 64-bit Python, ANSI C
+
+   4) ppro        - 32-bit Python, x86 CPU, PentiumPro or later
+
+   5) ansi32      - 32-bit Python, ANSI C
+
+   6) ansi-legacy - 32-bit Python, compiler without uint64_t
+
+   7) universal   - Mac OS only (multi-arch)
+
+
+It is possible to override autodetection by exporting:
+
+   PYTHON_DECIMAL_WITH_MACHINE=value, where value is one of the above options.
+
+
+NOTE
+====
+
+decimal.so is not built from a static libmpdec.a since doing so led to
+failures on AIX (user report) and Windows (mixing static and dynamic CRTs
+causes locale problems and more).
+
+
+

contrib/tools/python3/Modules/_decimal/libmpdec/README.txt

@@ -0,0 +1,88 @@
+
+
+libmpdec
+========
+
+libmpdec is a fast C/C++ library for correctly-rounded arbitrary precision
+decimal floating point arithmetic. It is a complete implementation of
+Mike Cowlishaw/IBM's General Decimal Arithmetic Specification.
+
+
+Files required for the Python _decimal module
+=============================================
+
+  Core files for small and medium precision arithmetic
+  ----------------------------------------------------
+
+    basearith.{c,h}  ->  Core arithmetic in base 10**9 or 10**19.
+    bits.h           ->  Portable detection of least/most significant one-bit.
+    constants.{c,h}  ->  Constants that are used in multiple files.
+    context.c        ->  Context functions.
+    io.{c,h}         ->  Conversions between mpd_t and ASCII strings,
+                         mpd_t formatting (allows UTF-8 fill character).
+    mpalloc.{c,h}    ->  Allocation handlers with overflow detection
+                         and functions for switching between static
+                         and dynamic mpd_t.
+    mpdecimal.{c,h}  ->  All (quiet) functions of the specification.
+    typearith.h      ->  Fast primitives for double word multiplication,
+                         division etc.
+
+    Visual Studio only:
+    ~~~~~~~~~~~~~~~~~~~
+      vcdiv64.asm   ->  Double word division used in typearith.h. VS 2008 does
+                        not allow inline asm for x64. Also, it does not provide
+                        an intrinsic for double word division.
+
+  Files for bignum arithmetic:
+  ----------------------------
+
+    The following files implement the Fast Number Theoretic Transform
+    used for multiplying coefficients with more than 1024 words (see
+    mpdecimal.c: _mpd_fntmul()).
+
+      umodarith.h        ->  Fast low level routines for unsigned modular arithmetic.
+      numbertheory.{c,h} ->  Routines for setting up the Number Theoretic Transform.
+      difradix2.{c,h}    ->  Decimation in frequency transform, used as the
+                             "base case" by the following three files:
+
+        fnt.{c,h}        ->  Transform arrays up to 4096 words.
+        sixstep.{c,h}    ->  Transform larger arrays of length 2**n.
+        fourstep.{c,h}   ->  Transform larger arrays of length 3 * 2**n.
+
+      convolute.{c,h}    ->  Fast convolution using one of the three transform
+                             functions.
+      transpose.{c,h}    ->  Transpositions needed for the sixstep algorithm.
+      crt.{c,h}          ->  Chinese Remainder Theorem: use information from three
+                             transforms modulo three different primes to get the
+                             final result.
+
+
+Pointers to literature, proofs and more
+=======================================
+
+  literature/
+  -----------
+
+    REFERENCES.txt  ->  List of relevant papers.
+    bignum.txt      ->  Explanation of the Fast Number Theoretic Transform (FNT).
+    fnt.py          ->  Verify constants used in the FNT; Python demo for the
+                        O(N**2) discrete transform.
+
+    matrix-transform.txt -> Proof for the Matrix Fourier Transform used in
+                            fourstep.c.
+    six-step.txt         -> Show that the algorithm used in sixstep.c is
+                            a variant of the Matrix Fourier Transform.
+    mulmod-64.txt        -> Proof for the mulmod64 algorithm from
+                            umodarith.h.
+    mulmod-ppro.txt      -> Proof for the x87 FPU modular multiplication
+                            from umodarith.h.
+    umodarith.lisp       -> ACL2 proofs for many functions from umodarith.h.
+
+
+Library Author
+==============
+
+  Stefan Krah <skrah@bytereef.org>
+
+
+

contrib/tools/python3/Modules/_hacl/README.md

@@ -0,0 +1,29 @@
+# Algorithm implementations used by the `hashlib` module.
+
+This code comes from the
+[HACL\*](https://github.com/hacl-star/hacl-star/) project.
+
+HACL\* is a cryptographic library that has been formally verified for memory
+safety, functional correctness, and secret independence.
+
+## Updating HACL*
+
+Use the `refresh.sh` script in this directory to pull in a new upstream code
+version.  The upstream git hash used for the most recent code pull is recorded
+in the script.  Modify the script as needed to bring in more if changes are
+needed based on upstream code refactoring.
+
+Never manually edit HACL\* files. Always add transformation shell code to the
+`refresh.sh` script to perform any necessary edits. If there are serious code
+changes needed, work with the upstream repository.
+
+## Local files
+
+1. `./include/python_hacl_namespaces.h`
+1. `./README.md`
+1. `./refresh.sh`
+
+## ACKS
+
+* Jonathan Protzenko aka [@msprotz on Github](https://github.com/msprotz)
+contributed our HACL\* based builtin code.

contrib/tools/python3/Modules/_xxtestfuzz/README.rst

@@ -0,0 +1,56 @@
+Fuzz Tests for CPython
+======================
+
+These fuzz tests are designed to be included in Google's `oss-fuzz`_ project.
+
+oss-fuzz works against a library exposing a function of the form
+``int LLVMFuzzerTestOneInput(const uint8_t* data, size_t length)``. We provide
+that library (``fuzzer.c``), and include a ``_fuzz`` module for testing with
+some toy values -- no fuzzing occurs in Python's test suite.
+
+oss-fuzz will regularly pull from CPython, discover all the tests in
+``fuzz_tests.txt``, and run them -- so adding a new test here means it will
+automatically be run in oss-fuzz, while also being smoke-tested as part of
+CPython's test suite.
+
+Adding a new fuzz test
+----------------------
+
+Add the test name on a new line in ``fuzz_tests.txt``.
+
+In ``fuzzer.c``, add a function to be run::
+
+    int $test_name (const char* data, size_t size) {
+        ...
+        return 0;
+    }
+
+
+And invoke it from ``LLVMFuzzerTestOneInput``::
+
+    #if _Py_FUZZ_YES(fuzz_builtin_float)
+        rv |= _run_fuzz(data, size, fuzz_builtin_float);
+    #endif
+
+``LLVMFuzzerTestOneInput`` will run in oss-fuzz, with each test in
+``fuzz_tests.txt`` run separately.
+
+Seed data (corpus) for the test can be provided in a subfolder called
+``<test_name>_corpus`` such as ``fuzz_json_loads_corpus``. A wide variety
+of good input samples allows the fuzzer to more easily explore a diverse
+set of paths and provides a better base to find buggy input from.
+
+Dictionaries of tokens (see oss-fuzz documentation for more details) can
+be placed in the ``dictionaries`` folder with the name of the test.
+For example, ``dictionaries/fuzz_json_loads.dict`` contains JSON tokens
+to guide the fuzzer.
+
+What makes a good fuzz test
+---------------------------
+
+Libraries written in C that might handle untrusted data are worthwhile. The
+more complex the logic (e.g. parsing), the more likely this is to be a useful
+fuzz test. See the existing examples for reference, and refer to the
+`oss-fuzz`_ docs.
+
+.. _oss-fuzz: https://github.com/google/oss-fuzz

contrib/tools/python3/Modules/cjkcodecs/README

@@ -0,0 +1,77 @@
+To generate or modify mapping headers
+-------------------------------------
+Mapping headers are generated from Tools/unicode/genmap_*.py
+
+
+
+Notes on implementation characteristics of each codecs
+-----------------------------------------------------
+
+1) Big5 codec
+
+  The big5 codec maps the following characters as cp950 does rather
+  than conforming Unicode.org's that maps to 0xFFFD.
+
+    BIG5        Unicode     Description
+
+    0xA15A      0x2574      SPACING UNDERSCORE
+    0xA1C3      0xFFE3      SPACING HEAVY OVERSCORE
+    0xA1C5      0x02CD      SPACING HEAVY UNDERSCORE
+    0xA1FE      0xFF0F      LT DIAG UP RIGHT TO LOW LEFT
+    0xA240      0xFF3C      LT DIAG UP LEFT TO LOW RIGHT
+    0xA2CC      0x5341      HANGZHOU NUMERAL TEN
+    0xA2CE      0x5345      HANGZHOU NUMERAL THIRTY
+
+  Because unicode 0x5341, 0x5345, 0xFF0F, 0xFF3C is mapped to another
+  big5 codes already, a roundtrip compatibility is not guaranteed for
+  them.
+
+
+2) cp932 codec
+
+  To conform to Windows's real mapping, cp932 codec maps the following
+  codepoints in addition of the official cp932 mapping.
+
+    CP932     Unicode     Description
+
+    0x80      0x80        UNDEFINED
+    0xA0      0xF8F0      UNDEFINED
+    0xFD      0xF8F1      UNDEFINED
+    0xFE      0xF8F2      UNDEFINED
+    0xFF      0xF8F3      UNDEFINED
+
+
+3) euc-jisx0213 codec
+
+  The euc-jisx0213 codec maps JIS X 0213 Plane 1 code 0x2140 into
+  unicode U+FF3C instead of U+005C as on unicode.org's mapping.
+  Because euc-jisx0213 has REVERSE SOLIDUS on 0x5c already and A140
+  is shown as a full width character, mapping to U+FF3C can make
+  more sense.
+
+  The euc-jisx0213 codec is enabled to decode JIS X 0212 codes on
+  codeset 2. Because JIS X 0212 and JIS X 0213 Plane 2 don't have
+  overlapped by each other, it doesn't bother standard conformations
+  (and JIS X 0213 Plane 2 is intended to use so.) On encoding
+  sessions, the codec will try to encode kanji characters in this
+  order:
+
+    JIS X 0213 Plane 1 -> JIS X 0213 Plane 2 -> JIS X 0212
+
+
+4) euc-jp codec
+
+  The euc-jp codec is a compatibility instance on these points:
+   - U+FF3C FULLWIDTH REVERSE SOLIDUS is mapped to EUC-JP A1C0 (vice versa)
+   - U+00A5 YEN SIGN is mapped to EUC-JP 0x5c. (one way)
+   - U+203E OVERLINE is mapped to EUC-JP 0x7e. (one way)
+
+
+5) shift-jis codec
+
+  The shift-jis codec is mapping 0x20-0x7e area to U+20-U+7E directly
+  instead of using JIS X 0201 for compatibility. The differences are:
+   - U+005C REVERSE SOLIDUS is mapped to SHIFT-JIS 0x5c.
+   - U+007E TILDE is mapped to SHIFT-JIS 0x7e.
+   - U+FF3C FULL-WIDTH REVERSE SOLIDUS is mapped to SHIFT-JIS 815f.
+

contrib/tools/python3/Python/README

@@ -0,0 +1 @@
+Miscellaneous source files for the main Python shared library

yt/yt/library/tracing/jaeger/tracer.cpp

@@ -559,6 +559,15 @@ void TJaegerTracer::Flush()
 
     auto config = Config_.Acquire();
 
+    auto flushStartTime = TInstant::Now();
+
+    if (OpenChannelConfig_ != config->CollectorChannelConfig) {
+        OpenChannelConfig_ = config->CollectorChannelConfig;
+        for (auto& [endpoint, channel] : CollectorChannels_) {
+            CollectorChannels_[endpoint].ForceReset(flushStartTime);
+        }
+    }
+
     DequeueAll(config);
 
     if (TInstant::Now() - LastSuccessfulFlushTime_ > config->QueueStallTimeout) {
@@ -573,15 +582,6 @@ void TJaegerTracer::Flush()
         return;
     }
 
-    auto flushStartTime = TInstant::Now();
-
-    if (OpenChannelConfig_ != config->CollectorChannelConfig) {
-        OpenChannelConfig_ = config->CollectorChannelConfig;
-        for (auto& [endpoint, channel] : CollectorChannels_) {
-            CollectorChannels_[endpoint].ForceReset(flushStartTime);
-        }
-    }
-
     std::stack<TString> toRemove;
     auto keys = ExtractKeys(BatchInfo_);