This website works better with JavaScript
Apache Arrow 5.0.0 (2021-07-22)
Bug Fixes
ARROW-6189 - [Rust] [Parquet] Plain encoded boolean column chunks limited to 2048 valuesARROW-6312 - [C++] Declare required Libs.private in arrow.pc package configARROW-7948 - [Go][Integration] Decimal integration failuresARROW-9594 - [Python] DictionaryArray.to_numpy does not correctly convert null indexes to null valuesARROW-10910 - [Python] Segmentation Fault when None given to read_table with legacy datasetARROW-10958 - [GLib] "Nested data conversions not implemented" through glib, but not through pyarrowARROW-11077 - [Rust] ParquetFileArrowReader panicks when trying to read nested listARROW-11146 - [CI][Python] Failing conda-python-3.8-jpype Nightly BuildARROW-11161 - [Python][C++] S3Filesystem: file Content-Type not set correctly?ARROW-11633 - [CI] [Documentation] Maven default skin not foundARROW-11780 - [C++][Python] StructArray.from_arrays() crashes Python interpreterARROW-11908 - [Rust] Intermittent Flight integration test failuresARROW-12007 - [C++] Loading parquet file returns "Invalid UTF8 payload" errorARROW-12055 - [R] is.na() evaluates to FALSE on Arrow NaN valuesARROW-12096 - [Python][C++] Pyarrow Parquet reader overflows INT96 timestamps when converting to Arrow Array (timestamp[ns])ARROW-12122 - [Python] Cannot install via pip M1 macARROW-12142 - [Python] undefined symbol: _ZN5arrow6StatusC1ENS_10StatusCodeERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEARROW-12150 - [Python] Bad type inference of mixed-precision DecimalsARROW-12232 - [Rust][Datafusion] Error with CAST: Unsupported SQL type TimeARROW-12240 - [Python] invalid-offsetof warning from apple clang-12ARROW-12377 - [Doc][Java] Java doc build brokenARROW-12407 - [Python] Deprecation warning when building PyArrowARROW-12431 - [Python] pa.array mask inverted when type is binary and value to be converted is numpy arrayARROW-12472 - [Python] read_table fails when passing a PEP519 filesystem objectARROW-12482 - [Doc][Python] Mention CSVStreamingReader pitfalls with type inferenceARROW-12491 - [Packaging] Required dependency on LZ4 >= 1.8 missing from CentOS RPM packagesARROW-12503 - [C++] Ensure using "lib/" for jemalloc's library directoryARROW-12508 - [R] expect_as_vector implementation causes test failure on R <= 3.3 and variables defined outside of test_that break build when no arrow installARROW-12543 - [CI][Python] Failing conda-python-3.9 Nightly BuildARROW-12568 - [Python][C++] Segfault when casting a sliced ListArray of int64 in v4.0.0ARROW-12569 - [R] [CI] Run revdep in CIARROW-12570 - [JS] Fix issues that blocked the v4.0.0 releaseARROW-12579 - [Python] Pyarrow 4.0.0 dependency numpy 1.19.4 throws errors on Apple silicon/M1 compilationARROW-12589 - [C++] Compiling on windows doesn't work when -DARROW_WITH_BACKTRACE=OFFARROW-12601 - [R][Packaging] Fix pkg-config check in r/configureARROW-12604 - [R][Packaging] Dataset, Parquet off in autobrew and CRAN Mac buildsARROW-12605 - [Documentation] Repair line numbers in dataset.rstARROW-12606 - [C++] Quantile and Mode functions failing on arrays with offsetARROW-12610 - [C++] Skip TestS3FSGeneric TestDeleteDir and TestDeleteDirContents on windows as they are flakyARROW-12611 - [CI][Python] Nightly test-conda-python-pandas-0.24 is failing due to numpy compat issueARROW-12613 - [Python] AttributeError when comparing a Scalar with NoneARROW-12614 - [C++][Compute] Revert support for Tables in ExecuteScalarExpressionARROW-12617 - [Python] pyarrow.orc.write_table signature reverses that of pyarrow.parquet.write_tableARROW-12620 - [C++] Dataset writing can only include projected columns if input columns are also includedARROW-12622 - [Python] Segfault when reading CSV inside Flight serverARROW-12630 - [Dev][Integration] conda-integration docker build failsARROW-12639 - [CI][Archery] Archery build fails to create branchARROW-12640 - [C++] Fix errors from VS 2019 in cpp/src/parquet/types.hARROW-12642 - [R] LIBARROW_MINIMAL, LIBARROW_DOWNLOAD, NOT_CRAN env vars should not be case-sensitiveARROW-12644 - [C++][Dataset] Support reading date/time-partitioned datasets accounting for URL encoding (Spark)ARROW-12646 - [C++][CI][Packaging][Python] Bump vcpkg version to its latest releaseARROW-12663 - [C++] segfault when arrow header is compiled with nvcc 11.2ARROW-12668 - [C++][Dataset] CountRows occasionally segfaultingARROW-12670 - [C++] extract_regex gives bizarre behavior after nulls or non-matchesARROW-12672 - [C++] Segfault casting result of "fill_null()" (not bitmap but unknown null_count)ARROW-12679 - [Java] JDBC adapter does not preserve SQL-nullabilityARROW-12684 - [Go][Flight] Fix nil dereference in error caseARROW-12708 - [C++] Valgrind errors when calling negate_checkedARROW-12729 - [R] Fix length method for Table, RecordBatchARROW-12746 - [Go][Flight] Client Auth handler overwrites outgoing metadataARROW-12756 - [C++] MSVC build fails with latest gtest from vcpkgARROW-12757 - [Dev][Archery] Warning about RUST variable in "archery docker run"ARROW-12762 - [Python] ListType doesn't preserve field name after pickle and unpickleARROW-12769 - [Python] Negative out of range slices yield invalid arraysARROW-12771 - [C++] Arrow compute hash_count skips following chunked arrays in streaming executionARROW-12772 - [CI] Merge script test fails due to missing dependencyARROW-12773 - [Docs] Clarify Java support for ORC and Parquet via JNI bindingsARROW-12774 - [C++][Compute] replace_substring_regex() creates invalid arrays => crashARROW-12776 - [Archery][Integration] Fix decimal case generation in write_js_test_jsonARROW-12779 - [Python][FlightRPC] Flight server segfaults with certain dataARROW-12780 - [CI][C++] MinGW builds failing when trying to build GandivaARROW-12790 - [Python] Cannot read from HDFS with blanks in path namesARROW-12793 - [Python] PYARROW_BUILD_TYPE=Debug does not work correctlyARROW-12797 - [JS] Update readme with new links and remove outdated examplesARROW-12798 - [JS] Use == null ComparisonARROW-12799 - [JS] Use Nullish Coalescing Operator (??) For DefaultsARROW-12804 - [C++] Array methods IsNull and IsValid is confused for NullTypeARROW-12807 - [C++] Fix merge conflicts with Future refactor/async IPCARROW-12838 - [Java][Gandiva] Fix JNI CI test for GandivaARROW-12842 - [Java][FlightRPC] Error metadata from FlightStatusException is not propagated to clientARROW-12850 - [R] is.nan() evaluates to null on Arrow null valuesARROW-12854 - [Dev][Release] Windows wheel verification script fails to download artifactsARROW-12857 - [C++] hash_aggregate_test not building on masterARROW-12864 - [C++] Remove needless out argument from arrow::internal::InvertBitmapARROW-12865 - [C++][Python] Python FlightRPC server cannot find RE2 symbolsARROW-12882 - [C++][Gandiva] Fix behavior of convevrt_replace function for empty replacement charARROW-12887 - [CI] AppVeyor pip install failure during setupARROW-12906 - [Python] `fill_null` called with a null value seg faults on non fixed-sized types.ARROW-12907 - [Java] Memory leak possible when exception reading from channel happensARROW-12911 - [Python] Export scalar aggregate options to pc.sum (sum of zero rows gives null; should give 0)ARROW-12917 - [C++][R][pyarrow] Failure importing some decimal types using the C data interfaceARROW-12918 - [C++] Build errors with Visual Studio 16.10.31321.278ARROW-12919 - [Developer Tools] Crossbow comment bot failing to react to commentsARROW-12935 - [C++][CI] Compiler error on some clang versionsARROW-12941 - [C++] csv reader skip_row does not properly update num_rows_seenARROW-12942 - [C++][Compute] The result of Arrow compute hash_min_max is incorrect if there are new groups in the subsequent chunksARROW-12956 - [C++] Fix crash on Parquet file (OSS-Fuzz)ARROW-12969 - [C++] match_substring doesn't match empty needle to empty haystackARROW-12974 - [R] test-r-without-arrow build fails because of example requiring ArrowARROW-12983 - [C++][Python] Converter::Extend gets stuck in infinite loop causing OOM if values don't fit in single chunkARROW-12987 - [CI] test-ubuntu-18.04 nightly builds are failing due to Gandiva "TestUpper" test failureARROW-12988 - [CI] The kartothek nightly integration build is failing (test_update_dataset_from_ddf_empty)ARROW-12988 - [CI] The kartothek nightly integration build is failing (test_update_dataset_from_ddf_empty)ARROW-12989 - [CI] "Dev PR" jobs undully cancelledARROW-12991 - [CI] Travis ARM builds often crashARROW-12993 - [Python] Address boundary error with invalid Feather file and stackprinterARROW-12995 - [C++] CSV reader should validate optionsARROW-12998 - [C++] Datasets needs dependency on xsimdARROW-13001 - [Go] Build failure in parquet/internal/bmi on s390xARROW-13003 - [C++] unaligned access in compute/exec/ cc filesARROW-13008 - [C++] Deprecation warning when compiling minimal exampleARROW-13010 - [C++][Compute] Support outputting to slices from kleene kernelsARROW-13018 - [C++][Docs] Use consistent terminology for nulls (min_count) in scalar aggregate kernelsARROW-13026 - [C++][CI] s390x job setup failsARROW-13037 - [R] Incorrect param when creating Expression crashes RARROW-13039 - [R] Fix error message handlingARROW-13041 - [C++] Unary kernels can leave uninitialized data under null entriesARROW-13046 - [Release] JS package failing test prior to publishARROW-13048 - [C++] S3FileSystem fails moving filepaths containing = or +ARROW-13053 - [Python] Build fails on MacOS Big Sur using homebrewed Arrow librariesARROW-13069 - [Website] Add Daniël to committer listARROW-13073 - [Developer] archery benchmark list: unexpected keyword 'benchmark_filter'ARROW-13080 - [Release] Generate the API docs in ubuntu 20.10ARROW-13083 - [Python] Wrong SCM version detection both in setup.py and crossbowARROW-13085 - [Python] Apache Arrow minimal cpp build segfaults with pyarrow libsARROW-13090 - [Python] Test failure with ffspec 2021.6.0ARROW-13104 - [C++] ByteStreamSplit implementation uses invalid pointer castARROW-13108 - [Python] Pyarrow 4.0.0 crashes upon import on macOS 10.13.6ARROW-13116 - [R] Test for RecordBatchReader to C-interface fails on arrow-r-minimal due to missing dependenciesARROW-13125 - [R] Throw error when 2+ args passed to desc() in arrange()ARROW-13128 - [C#] TimestampArray conversion logic for nano and micro is wrongARROW-13135 - [C++] Fix Status propagation in END_PARQUET_CATCH_EXCEPTIONSARROW-13139 - [C++] ReadaheadGenerator cannot be safely copied/movedARROW-13145 - [C++][CI] Flight test crashes on MinGWARROW-13148 - [Dev][Archery] Crossbow build submission failsARROW-13153 - [C++] `parquet_dataset` loses ordering of files in `_metadata`ARROW-13154 - [C++] Unions can not have 126 and 127 as type_codesARROW-13169 - [R] [C++] sorted partition keys can cause issuesARROW-13173 - [C++] TestAsyncUtil.ReadaheadFailed asserts occasionallyARROW-13187 - [c++][python] Possibly memory not deallocated when reading in CSVARROW-13189 - [R] Disable row-level metadata application on datasetsARROW-13203 - [R] Fix optional component checks causing failuresARROW-13207 - [Python][Doc] Dataset documentation still suggests deprecated scan method as the preferred iterative approachARROW-13216 - [R] Type checks test fails with rtools35ARROW-13217 - [C++][Gandiva] Correct convert_replace function for invalid chars on string beginningARROW-13223 - [C++][CI] Fix thread sanitizer failuresARROW-13225 - [Go][Flight] Implement Custom Middleware Interface and Enable Integration TestsARROW-13229 - [Python] ascii_trim, ascii_ltrim and ascii_rtrim lack optionsARROW-13239 - [Doc][Python] Dataset.head function doesn't mention required argumentARROW-13243 - [R] altrep function call in R 3.5ARROW-13246 - [C++] CSV skip_rows_after_names can discard data prematurallyARROW-13249 - [Java][CI] Consistent timeout in the Java JNI buildARROW-13253 - [C++][FlightRPC] Segfault when sending record batch >2GBARROW-13254 - [Python] Processes killed and semaphore objects leaked when reading pandas dataARROW-13265 - [R] cli valgrind errors in nightliesARROW-13266 - [JS] Improve benchmark names & add suite name to jsonARROW-13281 - [C++][Gandiva] Error on timestampDiffMonth function behavior for negative diff valuesARROW-13284 - [C++] Wrong pkg_check_modules() option nameARROW-13288 - [Python] Missing default values of kernel options in PyArrowARROW-13290 - Compilation fails on clang-12 and gcc-11 due to missing includeARROW-13305 - [C++] Unable to install nightly on Ubuntu 21.04 due to CSV optionsARROW-13315 - [R] Wrap r_task_group includes with ARROW_R_WITH_ARROW checkingARROW-13321 - [C++][Python] MakeArrayFromScalar doesn't work for FixedSizeBinaryTypeARROW-13324 - [R] Typo in bindings for utf8_reverse and ascii_reverseARROW-13332 - [C++] TSAN failure in TestAsyncUtil.ReadaheadFailedARROW-13341 - [C++] Segfault in arrow-compute-plan-test ExecPlanExecution.SourceScalarAggSinkARROW-13350 - [Python][CI] conda-python-3.7-pandas-0.24 nightly build failing in test_extract_datetime_componentsARROW-13352 - [C++] Valgrind failure in case_when kernelARROW-13353 - [Documentation] Build failing with sphinx.util.cfamily.DefinitionErrorARROW-13360 - [C++] Missing dependencies in C++ thirdparty offline dependencies versions.txtARROW-13363 - [R] is.nan() errors on non-floating point dataARROW-13368 - [C++][Doc] Rename project to make_struct in docsARROW-13381 - [C++] ArrayFromJSON doesn't work for float value dictionary typeARROW-13382 - [C++] Aggregation over scalars fails autobrew R jobARROW-13384 - [C++] Specify minimum required zstd version in cmakeARROW-13391 - [C++] CSV streaming reader does not include same error information as table readerARROW-13417 - [C++] The merged generator can sometimes pull from source sync-reentrantARROW-13419 - [JS] Fix perf testsARROW-13428 - [C++][Flight] -lssl is missing with bundled gRPC and system shared OpenSSLARROW-13431 - [Release] Bump go version to 1.15; don't verify rust source anymoreARROW-13432 - [Release] Fix ssh connection to the binary uploader container
New Features and Improvements
ARROW-2665 - [Python/C++] Add index() method to find first occurence of Python scalarARROW-3014 - [C++] Minimal writer adapter for ORC file formatARROW-3316 - [R] Multi-threaded conversion from R data.frame to Arrow table / record batchARROW-5385 - [Go] implement EXTENSION datatypeARROW-5640 - [Go] implement Map arrayARROW-6513 - [CI] The conda environment files arrow/ci/conda_env_*.yml should have .txt extensionARROW-6513 - [CI] The conda environment files arrow/ci/conda_env_*.yml should have .txt extensionARROW-7001 - [C++] Develop threading APIs to accommodate nested parallelismARROW-7114 - [JS][CI] NodeJS build fails on Github Actions Windows nodeARROW-7252 - [Rust] [Parquet] Reading UTF-8/JSON/ENUM field results in a lot of vec allocationARROW-7396 - [Format] Register media types (MIME types) for Apache Arrow formats to IANAARROW-8421 - [Rust] [Parquet] Implement parquet writerARROW-8459 - [Dev][Archery] Use a more recent cmake-formatARROW-8527 - [C++][CSV] Add support for ReadOptions::skip_rows >= block_sizeARROW-8655 - [C++][Dataset][Python][R] Preserve partitioning information for a discovered DatasetARROW-8676 - [Rust] Create implementation of IPC RecordBatch body buffer compression from ARROW-300ARROW-9054 - [C++] Add ScalarAggregateOptionsARROW-9056 - [C++] Support scalar aggregation over scalarsARROW-9140 - [R] Zero-copy Arrow to R where possibleARROW-9295 - [Archery] Support rust clippy in the lint commandARROW-9299 - [Python] Expose ORC metadata() in Python ORCFileARROW-9313 - [Rust] Use feature enumARROW-9421 - [C++][Parquet] Redundancies SchemaManifest::GetFieldIndicesARROW-9430 - [C++/Python] Kernel for SetItem(BooleanArray, values)ARROW-9697 - [C++][Dataset] num_rows method for Dataset/ScannerARROW-10031 - [Java] Support Java benchmark in ArcheryARROW-10115 - [C++] CSV empty quoted string is treated as NULLARROW-10316 - [Python] Consider using __wrapped__ for compute function introspectionARROW-10391 - [Rust] [Parquet] Nested Arrow readerARROW-10440 - [C++][Dataset][Python] Add a callback to visit file writers just before Finish()ARROW-10550 - [Rust] [Parquet] Write nested types (struct, list)ARROW-10557 - [C++] Add scalar string slicing/substring extract kernelARROW-10640 - [C++] An "if_else" kernel to combine two arrays based on a maskARROW-10658 - [Python][Packaging] Wheel builds for Apple SiliconARROW-10675 - [C++][Python] Support AWS S3 Web identity credentialsARROW-10797 - [C++] Investigate faster random generation for tests and benchmarksARROW-10926 - [Rust] Add parquet reader / writer for decimal typesARROW-10959 - [C++] Add scalar string join kernelARROW-11061 - [Rust] Validate array properties against schemaARROW-11173 - Add Map type as reader / writer in FieldReader / FieldWriterARROW-11199 - [C++][Python] Fix the unit tests for the ORC readerARROW-11206 - [C++][Compute][Python] Rename "project" kernel to "make_struct"ARROW-11342 - [Python] [Gandiva] Expose ToString and result type informationARROW-11499 - [Packaging] Remove all use of bintrayARROW-11514 - [R][C++] Bindings for paste(), paste0(), str_c()ARROW-11515 - [R] Bindings for strsplitARROW-11565 - [C++][Gandiva] Modify upper()/lower() to work with UTF8 and add INIT_CAP functionARROW-11608 - [CI] turbodbc integration tests are failing (build isue)ARROW-11660 - [C++] Move RecordBatch::SelectColumns method from R to C++ libraryARROW-11673 - [C++] Casting dictionary type to use different index typeARROW-11675 - [CI][C++] Resolve ctest failures on VS 2019 buildsARROW-11705 - [R] Support scalar value recycling in RecordBatch/Table$create()ARROW-11759 - [C++] Kernel to extract datetime components (year, month, day, etc) from timestamp typeARROW-11769 - [R] Pull groups from grouped_df into RecordBatch or TableARROW-11772 - [C++] Add asynchronous read to ipc::RecordBatchFileReaderARROW-11782 - [GLib][Ruby][Dataset] Remove bindings for internal classesARROW-11787 - [R] Implement write csvARROW-11843 - [C++] Add asynchronous read to parquet::arrow::FileReaderARROW-11849 - [R] Use roxygen @examplesIf tag in R docsARROW-11889 - [C++] Add parallelism to streaming CSV readerARROW-11909 - [C++] Get rid of MakeIteratorGeneratorARROW-11926 - [R] Pass on the new UCRT CRAN windows buildsARROW-11926 - [R] Pass on the new UCRT CRAN windows buildsARROW-11928 - [C++][Compute] Add ExecNode hierarchyARROW-11929 - [C++][Compute] Promote Expression to the compute namespaceARROW-11930 - [C++][Dataset][Compute] Refactor Dataset scans to use an ExecNode graphARROW-11932 - [C++] Provide ArrayBuilder::AppendScalarARROW-11950 - [C++][Compute] Add unary negative kernelARROW-11960 - [C++][Gandiva] Support escape in LIKEARROW-11980 - [Python] Remove "experimental" status from Table.replace_schema_metadataARROW-11986 - [C++][Gandiva] Implement IN expressions for doubles and floatsARROW-11990 - [C++][Compute] Use Status/Result return consistently to indicate errorsARROW-12004 - [C++] Result<detail::Empty> is annoyingARROW-12010 - [C++][Compute] Improve performance of the hash table used in GroupIdentifierARROW-12016 - [C++] Implement array_sort_indices and sort_indices for BOOL typeARROW-12050 - [C++][Python][FlightRPC] Use StopToken to enable interrupting long Flight operationsARROW-12074 - [C++][Compute] Add scalar arithmetic kernels for decimal inputsARROW-12083 - [R] schema use in open_datasetARROW-12166 - [C++][Gandiva] Implements CONVERT_TO(value, type) functionARROW-12184 - [R] Bindings for na.fail, na.omit, na.exclude, na.passARROW-12185 - [R] Bindings for any, allARROW-12198 - [R] bindings for strptimeARROW-12199 - [R] bindings for stddev, varianceARROW-12205 - [C++][Gandiva] Implement TO_TIME([number] secs) and TO_TIMESTAMP([number] secs) functionARROW-12231 - [C++][Dataset] Separate datasets backed by readers from InMemoryDatasetARROW-12253 - [Rust] [Ballista] Implement scalable joinsARROW-12255 - [Rust] [Ballista] Integrate scheduler with DataFusionARROW-12256 - [Rust] [Ballista] Add DataFrame supportARROW-12257 - [Rust] [Ballista] Publish user guide to Arrow siteARROW-12261 - [Rust] [Ballista] Ballista should not have its own DataFrame APIARROW-12291 - [R] Determine the type of an unevaluated expressionARROW-12310 - [Java] ValueVector#getObject should support covariance for complex typesARROW-12355 - [C++] Implement efficient async CSV scanningARROW-12362 - [Rust] [DataFusion] topk_query test failureARROW-12364 - [Python] [Dataset] Add metadata_collector option to ds.write_dataset()ARROW-12378 - [C++][Gandiva] Implement castVARBINARY functionsARROW-12386 - [C++] Support file parallelism in AsyncScannerARROW-12391 - [Rust][DataFusion] Implement date_trunc() functionARROW-12392 - [C++] Restore asynchronous streaming CSV readerARROW-12393 - [JS] Optimally use closure compilerARROW-12403 - [Rust] [Ballista] Integration tests should check that query results are correctARROW-12415 - [CI] [Python] ERROR: Failed building wheel for pygit2 on ARM64ARROW-12424 - [Go][Parquet] Add Schema PackageARROW-12428 - [Python] pyarrow.parquet.read_* should use pre_buffer=TrueARROW-12434 - [Rust] [Ballista] Show executed plans with metricsARROW-12442 - [CI] Set job timeouts on GitHub ActionsARROW-12443 - [C++][Gandiva] Implement castVARCHAR function for binary inputARROW-12444 - [RUST] [CI] Remove Rust and point integration tests to arrow-rs repoARROW-12445 - [Rust] Design and implement packaging process to bundle Rust in signed tarARROW-12468 - [Python][R] Expose UseAsync to python/RARROW-12478 - [C++] Support LLVM 12ARROW-12484 - [CI] Change jinja macros to not require CROSSBOW_TOKEN to upload artifacts in Github ActionsARROW-12489 - [Developer] autotune is brokenARROW-12490 - [Dev] Use miniforge for all platformsARROW-12492 - [Python] Add an helper method to decode a DictionaryArray back to a plain ArrayARROW-12496 - [C++][Dataset] Ensure Scanner tests fully cover asyncARROW-12499 - [C++][Compute][R] Add ScalarAggregateOptions to Any and All kernelsARROW-12500 - [C++][Dataset] Consolidate similar tests for file formatsARROW-12501 - [CI][Ruby] Remove needless workaround for MinGW buildARROW-12507 - [CI] Remove duplicated cron/nightly buildsARROW-12512 - [C++][Dataset] Implement CSV writing supportARROW-12514 - [Release] Don't run Gandiva related Ruby test with ARROW_GANDIVA=OFFARROW-12517 - [Go] Expose App Metadata in Flight clientARROW-12518 - [Python] Expose Parquet statistics has_null_count / has_distinct_countARROW-12520 - [R] Minor docs updatesARROW-12522 - [C++] Implement asynchronous/"lazy" variants of ReadRangeCacheARROW-12525 - [JS] Vector toJSON returns an arrayARROW-12527 - [Dev] Don't try getting JIRA information for MINOR PRARROW-12528 - [JS] Support typed arrays in Table.newARROW-12530 - [C++] Remove Buffer::mutable_data_ member and use const_cast on data_ only if is_mutable_ is trueARROW-12533 - [C++] Random real generator is slow on Arm64 Linux when built with clangARROW-12534 - [C++][Gandiva] Implement LEFT and RIGHT functions on Gandiva for string input valuesARROW-12537 - [JS] Docs build should not include test sourcesARROW-12541 - [Docs] Improve styling/readability of tables in the new doc themeARROW-12551 - [Java][Release] Java post-release tests fail due to missing testing dataARROW-12554 - Allow duplicates in the value_set for compute::is_inARROW-12555 - [Java][Release] Java post-release script misses dataset JNI bindingsARROW-12556 - [C++][Gandiva] Implement BYTESUBSTRING functions on GandivaARROW-12560 - [C++] Investigate utilizing aggressive thread task creation when adding callback to finished futureARROW-12567 - [C++][Gandiva] Implement LPAD and RPAD functions for string input valuesARROW-12567 - [C++][Gandiva] Implement LPAD and RPAD functions for string input valuesARROW-12571 - [R][CI] Run nightly R with valgrindARROW-12575 - [R] Use unary negative kernelARROW-12577 - [Website] Use Artifactory instead of Bintray in all placesARROW-12578 - [JS] Simplify UTF8 handling in NodeJSARROW-12581 - [C++][FlightRPC] Benchmark compression with real dataARROW-12584 - [C++][Python] Expose method for benchmarking tools to release unused memory from the allocatorsARROW-12591 - [Java][Gandiva] Create single Gandiva jar for MacOS and LinuxARROW-12593 - [Packaging][Ubuntu] Add support for Ubuntu 21.04ARROW-12597 - [C++] Implement OptionalParallelForAsyncARROW-12598 - [C++][Dataset] Implement row-count for CSV or allow selecting 0 columns from CSVARROW-12599 - [Doc][Python] Documentation missing for pyarrow.TableARROW-12600 - [CI] Push docker images from crossbow tasksARROW-12602 - [R] Add BuildInfo from C++ to arrow_infoARROW-12608 - [C++] Add split_pattern_regex functionARROW-12612 - [C++][Compute] Add Expression to type_fwd.hARROW-12619 - [Python] pyarrow sdist should not require gitARROW-12621 - [C++][Gandiva] Add alias to sha1 and sha256 functionsARROW-12631 - [Python] pyarrow.dataset.write_table should accept a Scanner to writeARROW-12643 - Add documentation for experimental reposARROW-12645 - [Python] Fix numpydoc validationARROW-12648 - [C++][FlightRPC] Allow using TLS in benchmarkARROW-12649 - [Python/Packaging] Move conda-aarch64 to Azure with cross-compilationARROW-12653 - [Archery] allow me to add a comment to crossbow requestsARROW-12658 - [C++] Bump aws-c-common to v0.5.10ARROW-12660 - [R] Post-4.0 adjustments for CRANARROW-12661 - [C++] CSV add skip rows after column namesARROW-12662 - [Website] Force to use squash mergeARROW-12667 - [Python] Ensure test coverage for conversion of strided numpy arraysARROW-12675 - [C++] CSV should include line/row numbers in parsing error messagesARROW-12677 - [Python] Add a mask argument to pyarrow.StructArray.from_arraysARROW-12685 - [C++][Compute] Add unary absolute value kernelARROW-12686 - [C++][Python][FlightRPC] Support export_to_c in DoGet/inherit from RecordBatchReaderARROW-12687 - [C++][Python][Dataset] Support C Data Interface with ScannerARROW-12689 - [R] Implement ArrowArrayStream C interfaceARROW-12692 - [R] Improve tests and comments for strsplit() bindingsARROW-12694 - [R][CI] rtools35 job failing on 32-bit build testsARROW-12696 - [R] Improve testing of error messages converted to warningsARROW-12699 - [CI][Packaging][Java] Generate a jar compatible with Linux and MacOS for all Arrow componentsARROW-12701 - [Website][Release] Include Rust and DataFusion commits, contributors, changes in release notesARROW-12702 - [JS] Upgrade Webpack and terserARROW-12703 - [JS] Separate Table from DataFrameARROW-12704 - [JS] use optional chainingARROW-12709 - [C++] Add variadic string join kernelARROW-12713 - [C++] String reverse kernelARROW-12715 - [C++] SQL-style glob string match kernelARROW-12716 - [C++] Left/right/center string padding kernelsARROW-12717 - [C++] Substring find position kernelARROW-12719 - [C++][Python] pyarrow.fs.S3FileSystem pass extra kwargs i.e ACLARROW-12721 - [CI] Fix path for uploading aarch64 conda artifacts from the nightly buildsARROW-12722 - [R] Raise error when attemping to print table with duplicated namingARROW-12730 - [MATLAB] Update featherreadmex and featherwritemex to build against latest arrow c++ APIsARROW-12731 - [R] Use InMemoryDataset for Table/RecordBatch in dplyr codeARROW-12736 - [C++] Eliminate unnecessary copy in FieldPath::Get()ARROW-12738 - [CI] [Gandiva] Nightly build error in azure-conda-osx-clang-py38 (and py39, py*-r*)ARROW-12741 - [CI] Configure GitHub Token for Nightly BuildsARROW-12745 - [C++][Compute] Add floor, ceiling, and truncate kernelsARROW-12749 - [C++] Unnecessary copy cause by constructing RecordBatch/Table/Schema from lvaluesARROW-12750 - [CI] [R] Actually pass parameterized docker options to the templatesARROW-12751 - [C++] Add variadic row-wise min/max kernels (least/greatest)ARROW-12758 - [R] Add examples to more function documentationARROW-12760 - [C++][Python][R] S3FileSystem: IO thread parallelism limited to 8 threadsARROW-12761 - [R] Better error handling for write_to_rawARROW-12764 - [CI] Fix arguments in Conda Windows buildsARROW-12777 - [R] Convert all inputs to Arrow objects in match_arrow and is_inARROW-12781 - [R] Implement is.type() functions for dplyrARROW-12785 - [CI] the r-devdocs build errors when brew installing gccARROW-12791 - [R] Better error handling for DatasetFactory$Finish() when no format specifiedARROW-12796 - [JS] Support JSON output from benchmarksARROW-12800 - [JS] Drop IE Support and remove text encoder and decoder polyfillsARROW-12801 - [CI][Packaging][Java] Include all modules in script that generate Arrow jarsARROW-12806 - [Python] test_write_to_dataset_filesystem missing a dataset markARROW-12808 - [JS] Document browser supportARROW-12810 - [Python] Run tests with AWS_EC2_METADATA_DISABLED=trueARROW-12812 - [Packaging][Java] Improve JNI jars buildARROW-12824 - [R][CI] Upgrade builds for R 4.1 releaseARROW-12827 - [C++] [Dataset] Review error pass-through in the datasets APIARROW-12829 - [GLib][Ruby] Add support for Apache Arrow FlightARROW-12831 - [CI][macOS] Remove needless Homebrew workaroundARROW-12832 - [JS] Write benchmarks in TypeScriptARROW-12833 - [JS] Construct perf data in JSARROW-12835 - [C++] Implement case insenstive match in match_substring(_regex) and match_likeARROW-12836 - [C++] Installation on IBM i fails because of CxxFlagsARROW-12841 - [R] Add examples to more function documentation - part 2ARROW-12843 - [C++][Compute] Add is_inf kernel for floating point arraysARROW-12848 - [Release] Mail template points to 404ARROW-12851 - [Go][Parquet] Add Encoding Package Part 1ARROW-12856 - [C++][Gandiva] Implement castBIT and castBOOLEAN functions on GandivaARROW-12859 - [C++] Add ScalarFromJSON for easier testingARROW-12861 - [C++][Compute] Add sign function kernelsARROW-12867 - [R] Bindings for abs()ARROW-12868 - [R] Bindings for find_substring and find_substring_regexARROW-12869 - [R] Bindings for utf8_reverse and ascii_reverseARROW-12870 - [R] Bindings for stringr::str_likeARROW-12875 - [JS] Upgrade Jest and other minor updatesARROW-12883 - [R] [CI] version compatibility fails on R 4.1ARROW-12891 - [C++][Compute][Dataset] Extract subtree pruning logic to compute::ARROW-12894 - [R] Bump R versionARROW-12895 - [CI] Use "concurrency" setting on Github ActionsARROW-12898 - [Release][C#] Package upload script is brokenARROW-12900 - [Python][Documentation] an np import in Reading Datasets docsARROW-12901 - [R] Follow on to more examplesARROW-12909 - [R][Release] Build of ubuntu-docs is failingARROW-12912 - [Website] Use .asf.yaml for publishingARROW-12915 - [Release] Build of ubuntu-docs is failing on thriftARROW-12936 - [C++][Gandiva] Implement ASCII Hive function on GandivaARROW-12937 - [C++] Allow specifying default metadata for new S3 filesARROW-12939 - [R] Simplify RTask stop handlingARROW-12940 - [R] Expose C interface as R6 methodsARROW-12948 - [C++] Add string slice replace kernelARROW-12949 - [C++] Add string starts-with/ends-with kernelsARROW-12950 - [C++] Add substring count kernelARROW-12951 - [C++] Refactor StringTransformARROW-12952 - [C++] Add regex count kernelARROW-12955 - [C++] Add additional type support for if_else kernelARROW-12957 - [R] rchk issues on cranARROW-12961 - [C++] MSVC issues warning building PyArrow on WindowsARROW-12962 - [GLib][Ruby] Add Arrow:ScalarARROW-12964 - [R] Add bindings for ifelse() and if_else()ARROW-12966 - [Python] Expose Python binding for ElementWiseAggregateOptionsARROW-12967 - [R] Add bindings for pmin() and pmax()ARROW-12968 - [R] [CI] Add an rchk job to our nightliesARROW-12972 - [CI] ][C++] archive_write_add_filter_zstd error on CentOS + ARM64ARROW-12975 - [C++][Python] if_else kernel doesn't support upcastingARROW-12982 - [C++] Re-enable unused-variable warningARROW-12984 - [C++] Passing options parameter of Count/Index aggregation by referenceARROW-12985 - [Python][Packaging] Unable to install pygit2 in the arm64 wheel buildsARROW-12986 - [C++][Gandiva] Implement new cache eviction policy in GandivaARROW-12992 - [R] bindings for substr(), substring(), str_sub()ARROW-12994 - [R] Fix tests that assume UTC local tzARROW-12996 - [C++] CSV stream reader has no progress indicationARROW-13002 - [C++] Add a check for the utf8proc's version in CMakeARROW-13005 - [C++] Support filter/take for union data type.ARROW-13006 - [C++][Gandiva] Implement BASE64 and UNBASE64 Hive functions on GandivaARROW-13009 - [Doc][Dev] Document builds mailing-listARROW-13022 - [R] bindings for lubridate's year, isoyear, quarter, month, day, wday, yday, isoweek, hour, minute, and second functionsARROW-13025 - [C++][Compute] Enhance FunctionOptions with equality, debug representability, and serializabilityARROW-13027 - [C++] Fix ASAN stack traces in CIARROW-13030 - [CI][Go] Setup Arm64 golang CIARROW-13031 - [JS] Support arm in closure compiler on macOSARROW-13032 - [Java] Update gauva versionARROW-13034 - [Python][Docs] Update outdated examples for hdfs/azure on the Parquet doc pageARROW-13036 - [Doc] Mention recommended file extension(s) for Arrow IPCARROW-13042 - [C++] Automatic checks that kernels don't leave uninitialized data in outputARROW-13043 - [GLib][Ruby] Add GArrowEqualOptionsARROW-13044 - [Java] Union vectors should extend ValueVectorARROW-13045 - [Packaging][RPM][deb] Don't install system utf8proc if it's oldARROW-13047 - [Website] Add kiszk to committer listARROW-13049 - [C++][Gandiva] Implement BIN Hive function on GandivaARROW-13050 - [C++][Gandiva] Implement SPACE Hive function on GandivaARROW-13054 - [C++] Add option to specify the first day of the week for the "day_of_week" temporal kernelARROW-13064 - [C++] Add a general "if, ifelse, ..., else" kernel ("CASE WHEN")ARROW-13065 - [Packaging][RPM] Add missing required LZ4 version informationARROW-13068 - [GLib][Dataset] Change prefix to gadataset_ from gad_ARROW-13070 - [R] bindings for sd and varARROW-13072 - [C++] Add bitwise arithmetic compute functionsARROW-13074 - [Python] Start with deprecating ParquetDataset custom attributesARROW-13075 - [Python] Expose C data interface API for pyarrow.FieldARROW-13076 - [Java] Enable ExtensionType to use StructVector and UnionVector for underlying storageARROW-13082 - [CI] Forward R argument to ubuntu-docs buildARROW-13086 - [Python] Expose Parquet ArrowReaderProperties::coerce_int96_timestamp_unit_ARROW-13086 - [Python] Expose Parquet ArrowReaderProperties::coerce_int96_timestamp_unit_ARROW-13091 - [Python] Add compression_level argument to IpcWriteOptions constructorARROW-13092 - [C++] CreateDir should fail if the target exists and is not a directoryARROW-13095 - [C++] Implement trigonometric compute functionsARROW-13096 - [C++] Implement logarithm compute functionsARROW-13097 - [C++] Provide a simple reflection utility for {{struct}}sARROW-13098 - [Dev][Archery] Reorganize docker submodule to its own subpackageARROW-13100 - [MATLAB] Integrate GoogleTest with MATLAB Interface C++ CodeARROW-13101 - [Python][Doc] pyarrow.FixedSizeListArray does not appear in the documentationARROW-13110 - [C++] Deadlock can happen when using BackgroundGenerator without transferring callbacksARROW-13113 - [R] use RTasks to manage parallel in converting arrow to RARROW-13117 - [R] Retain schema in new ExpressionsARROW-13119 - [R] Set empty schema in scalar ExpressionsARROW-13124 - [Ruby] Add support for memory viewARROW-13127 - [R] Valgrind nightly errorsARROW-13136 - [C++] Add a "coalesce" variadic scalar kernelARROW-13137 - [C++][Documentation] Make in-table references consistentARROW-13140 - [C++/Python] Upgrade libthrift pin in the nightliesARROW-13142 - [Python] Use vector append when converting from list of non-strided numpy arraysARROW-13147 - [Java] Respect the rounding policy when allocating vector buffersARROW-13157 - [C++] Add find_substring_regex kernel and implement ignore_case for find_substringARROW-13158 - [Python] Fix repr and contains of StructScalar with duplicate field namesARROW-13162 - [C++][Gandiva] Add new alias for extract date functions in Gandiva registryARROW-13171 - [R] Add binding for str_pad()ARROW-13190 - [C++] [Gandiva] Change behavior of INITCAP functionARROW-13194 - [Java][Document] Create prose document about Java algorithmsARROW-13195 - [R] Problem with rlang reverse dependency checksARROW-13199 - [R] add ubuntu 21.04 to nightly buildsARROW-13200 - [R] Add binding for case_when()ARROW-13201 - [R] Add binding for coalesce()ARROW-13210 - [Python][CI] Fix vcpkg caching mechanism for the macOS wheelsARROW-13211 - [C++][CI] Remove outdated Github Actions ARM buildsARROW-13212 - [Release] Support deploying to test PyPI in the python post release scriptARROW-13215 - [R] [CI] Add ENV TZ to docker filesARROW-13218 - [Doc] Document/clarify conventions for timestamp storageARROW-13219 - [C++][GLib] Demote/deprecate CompareOptionsARROW-13224 - [Python][Doc] Documentation missing for pyarrow.dataset.write_datasetARROW-13226 - [Python] Add a general purpose cython trampolining utilityARROW-13228 - [C++] S3 CreateBucket fails because AWS treats us-east-1 differently than other regionsARROW-13230 - Add CSV Writer documentationARROW-13234 - [C++] Add string padding option to determine which side the extra space goes onARROW-13235 - [C++] Make type_name equal to options class name for all FunctionOptionTypesARROW-13236 - [Python] Improve repr of pyarrow.compute.FunctionOptionsARROW-13238 - [C++][Dataset][Compute] Substitute ExecPlan impl for dataset scansARROW-13242 - [C++] Improve decimal random generationARROW-13244 - [C++] Add facility to get current thread idARROW-13258 - [Python] Improve the repr of ParquetFileFragmentARROW-13262 - [R] transmute() fails after pulling data into RARROW-13273 - [C++] Don't use .pc only in CMake paths for Requires.privateARROW-13274 - [JS] Remove WebpackARROW-13275 - [JS] Fix perf testsARROW-13276 - [GLib][Ruby][Flight] Add support for ListFlightsARROW-13277 - [JS] Add declaration mapsARROW-13280 - [R] Bindings for log and trig functionsARROW-13282 - [C++] Remove obsolete generated filesARROW-13283 - [Developer Tools] Support passing through memory limits in archery docker runARROW-13286 - [CI] Require docker-compose 1.27.0 or laterARROW-13289 - [C++] Log functions don't have int kernelsARROW-13291 - [GLib][CI] Require gobject-introspection 3.4.5 or laterARROW-13296 - [C++] Provide reflection-compatible enum replacementARROW-13299 - [JS] Upgrade ix and rxjsARROW-13303 - [JS] Revise bundlesARROW-13306 - [Java][JDBC] use ResultSetMetaData.getColumnLabel instead of ResultSetMetaData.getColumnNameARROW-13313 - [C++][Compute] Add ScalarAggregateNodeARROW-13320 - [Website] Add MIME types to FAQARROW-13323 - [Archery] Validate docker compose configurationARROW-13343 - [R] Update NEWS.md for 5.0ARROW-13346 - [C++] Remove compile time parsing from EnumTypeARROW-13355 - [R] ensure that sf is installed in our revdep jobARROW-13357 - [R] bindings for sign()ARROW-13365 - [R] bindings for floor/ceiling/truncateARROW-13385 - [C++][Compute] Document out-of-source addition to the FunctionRegistryARROW-13386 - [R][C++] CSV streaming changes break Rtools 35 32-bit buildARROW-13418 - [R] typo in python.rPARQUET-1798 - [C++] Review logic around automatic assignment of field_id'sPARQUET-1998 - [C++] Implement LZ4_RAW compressionPARQUET-2056 - [C++] Add ability for retrieving dictionary and indices separately for ColumnReader
Apache Arrow 3.0.0 (2021-01-18)
New Features and Improvements
ARROW-1846 - [C++] Implement "any" reduction kernel for boolean dataARROW-3850 - [Python] Support MapType and StructType for enhanced PySpark integrationARROW-4193 - [Rust] Add support for decimal data typeARROW-4544 - [Rust] Read nested JSON structs into StructArraysARROW-4804 - [Rust] Read temporal values from CSV - Parse Date32 and Date64 in CSV readerARROW-4960 - [R] Add crossbow task for r-arrow-feedstockARROW-4970 - [C++][Parquet] Implement parquet::FileMetaData::EqualsARROW-5336 - [C++] Implement arrow::Concatenate for dictionary-encoded arrays with unequal dictionariesARROW-5350 - [Rust] Support filtering on primitive/string listsARROW-5394 - [C++] Benchmarks for IsIn KernelARROW-5679 - [Python] Drop Python 3.5 from support matrixARROW-5950 - [Rust] [DataFusion] Add ability to log via logger dependencyARROW-6071 - [C++] Implement casting Binary <-> LargeBinaryARROW-6697 - [Rust] [DataFusion] Validate that all parquet partitions have the same schemaARROW-6715 - [Website] Describe "non-free" component is needed for Plasma packages in install pageARROW-6883 - [C++] Support sending delta DictionaryBatch or replacement DictionaryBatch in IPC stream writer classARROW-6995 - [Packaging][Crossbow] The windows conda artifacts are not uploaded to GitHub releasesARROW-7531 - [C++] Investigate header cost reductionARROW-7800 - [Python] Expose GetRecordBatchReader API in PyArrowARROW-7842 - [Rust] [Parquet] Implement array reader for list typeARROW-8113 - [C++] Implement a lighter-weight variantARROW-8199 - [C++] Add support for multi-column sort on TableARROW-8289 - [Rust] [Parquet] Implement minimal Arrow Parquet writer as starting point for full writerARROW-8423 - [Rust] [Parquet] Serialize arrow schema into metadata when writing parquetARROW-8425 - [Rust] [Parquet] Add support for writing temporal typesARROW-8426 - [Rust] [Parquet] Add support for writing dictionary typesARROW-8426 - [Rust] [Parquet] Add support for writing dictionary typesARROW-8853 - [Rust] [Integration Testing] Enable Flight testsARROW-8876 - [C++] Implement casts from date types to TimestampARROW-8883 - [Rust] [Integration Testing] Enable passing tests and update spec docARROW-9001 - [R] Box outputs as correct type in call_functionARROW-9164 - [C++] Provide APIs for adding "docstrings" to arrow::compute::Function classes that can be accessed by bindingsARROW-9187 - [R] Add bindings for arithmetic kernelsARROW-9296 - [CI][Rust] Enable more clippy lint checksARROW-9304 - [C++] Add "AppendEmptyValue" builder APIs for use inside StructBuilder::AppendNullARROW-9361 - [Rust] Move other array types into their own modulesARROW-9400 - [Python] Do not depend on conda-forge static libraries in Windows wheel buildsARROW-9475 - [Java] Clean up usages of BaseAllocator, use BufferAllocator insteadARROW-9489 - [C++] Add fill_null kernel implementation for (array[string], scalar[string])ARROW-9555 - [Rust] [DataFusion] Add inner (hash) equijoin physical planARROW-9564 - [Packaging] Vendor r-arrow-feedstock conda-forge recipeARROW-9674 - [Rust] Parquet reader should implement Send + SyncARROW-9704 - [Java] TestEndianness.testLittleEndian fails on big endian platformARROW-9707 - [Rust] [DataFusion] Re-implement threading modelARROW-9709 - [Java] Test cases in arrow-vector assume little-endian platformARROW-9728 - [Rust] [Parquet] Compute nested definition and repetition for structsARROW-9747 - [C++][Java][Format] Support Decimal256 TypeARROW-9771 - [Rust] [DataFusion] Predicate Pushdown Improvement: treat predicates separated by AND separatelyARROW-9803 - [Go] Add initial support for s390xARROW-9804 - [FlightRPC] Authentication RedesignARROW-9828 - [Rust] [DataFusion] TableProvider trait should support predicate push-downARROW-9861 - [Java] Failed Arrow Vector on big-endian platformARROW-9862 - Throw an exception in UnsafeDirectLittleEndian on Big-Endian platformARROW-9911 - [Rust][DataFusion] SELECT with no FROM clause should produce a single row of output
ARROW-9945 - [C++][Dataset] Refactor Expression::Assume to return a ResultARROW-9991 - [C++] split kernels for strings/binaryARROW-10002 - [Rust] Trait-specialization requires nightlyARROW-10021 - [C++][Compute] Support finding nth frequently used value in mode kernelARROW-10032 - [Documentation] C++ Windows docs are out of dateARROW-10079 - [Rust]: Benchmark and improve count_set_bits functionARROW-10095 - [Rust] [Parquet] Update for IPC changesARROW-10097 - [C++] Persist SetLookupState in between usages of IsIn when filtering dataset batchesARROW-10106 - [FlightRPC][Java] Expose onIsReady() callback on OutboundStreamListenerARROW-10108 - [Rust] [Parquet] Fix compiler warning about unused return valueARROW-10109 - [Rust] Add support to produce a C Data interfaceARROW-10110 - [Rust] Add support to consume C Data InterfaceARROW-10131 - [C++][Dataset] Lazily parse parquet metadata / statistics in ParquetDatasetFactory and ParquetFileFragmentARROW-10135 - [Rust] [Parquet] Refactor file module to help adding sourcesARROW-10143 - [C++] ArrayRangeEquals should accept EqualOptionsARROW-10144 - [Flight] Add support for using the TLS_SNI extensionARROW-10149 - [Rust] Add support to external release of un-owned buffersARROW-10163 - [Rust] [DataFusion] Add DictionaryArray coercion supportARROW-10168 - [Rust] [Parquet] Extend arrow schema conversion to projected fieldsARROW-10173 - [Rust][DataFusion] Improve performance of equality to a constant predicate supportARROW-10180 - [C++][Doc] Update dependency management docs following aws-sdk-cpp additionARROW-10182 - [C++] Add basic continuation support to futuresARROW-10191 - [Rust] [Parquet] Add roundtrip tests for single column batchesARROW-10197 - [Gandiva][python] Execute expression on filtered dataARROW-10203 - [Doc] Capture guidance for endianness support in contributors guide.ARROW-10207 - [C++] Unary kernels that results in a list have no preallocated offset bufferARROW-10208 - [C++] String split kernels do not propagate nulls correctly on sliced inputARROW-10216 - [Rust] Simd implementation of min/max aggregation kernels for primitive typesARROW-10224 - [Python] Add support for Python 3.9 except macOS wheel and Windows wheelARROW-10225 - [Rust] [Parquet] Fix null bitmap comparisons in roundtrip testsARROW-10228 - [Julia] Donate Julia ImplementationARROW-10236 - [Rust] [DataFusion] Make DataFusion casting rules consistent with cast kernelARROW-10241 - [C++][Compute] Add variance kernel benchmarkARROW-10249 - [Rust]: Support Dictionary types for ListArrays in arrow json readerARROW-10259 - [Rust] Support field metadataARROW-10261 - [Rust] [BREAKING] Lists should take Field instead of DataTypeARROW-10263 - [C++][Compute] Improve numerical stability of variances mergingARROW-10268 - [Rust] Support writing dictionaries to IPC file and streamARROW-10269 - [Rust] Update nightly: Oct 2020 EditionARROW-10277 - [C++] Support comparing scalars approximatelyARROW-10289 - [Rust] Support reading dictionary streamsARROW-10292 - [Rust] [DataFusion] Simplify mergeARROW-10295 - [Rust] [DataFusion] Simplify accumulatorsARROW-10300 - [Rust] Improve benchmark documentation for generating/converting TPC-H dataARROW-10301 - [C++] Add "all" boolean reducing kernelARROW-10302 - [Python] Don't double-package plasma-store-serverARROW-10304 - [C++][Compute] Optimize variance kernel for integersARROW-10310 - [C++][Gandiva] Add single argument round() in GandivaARROW-10311 - [Release] Update crossbow verification processARROW-10313 - [C++] Improve UTF8 validation speed and CSV string conversionARROW-10318 - [C++] Use pimpl idiom in CSV parserARROW-10319 - [Flight][Go] Add Context to Client Auth Handler functions for FlightARROW-10320 - [Rust] Convert RecordBatchIterator to a StreamARROW-10322 - [C++][Dataset] Minimize Expression to a wrapper around compute::FunctionARROW-10323 - [Release][wheel] Add missing verification setup stepARROW-10325 - [C++][Compute] Separate aggregate kernel registrationARROW-10328 - [C++] Consider using fast-double-parserARROW-10330 - [Rust][Datafusion] Implement nullif() function for DataFusionARROW-10331 - [Rust] [DataFusion] Re-organize errorsARROW-10332 - [Rust] Allow CSV reader to start from a lineARROW-10334 - [Rust] [Parquet] Support reading and writing Arrow NullArrayARROW-10336 - [Rust] Support fromIter and toIter for stringsARROW-10337 - [C++] More liberal parsing of ISO8601 timestamps with fractional secondsARROW-10338 - [Rust]: Use const fn for applicable methodsARROW-10340 - [Packaging][deb][RPM] Use Python 3.8 for pygit2ARROW-10356 - [Rust] [DataFusion] Add support for is_inARROW-10363 - [Python] Remove workaround for CMake bug in manylinuxARROW-10366 - [Rust] [DataFusion] Remove collect from mergeARROW-10375 - [Rust] Remove PrimitiveArrayOpsARROW-10378 - [Rust] Update take() kernel with support for large listsARROW-10381 - [Rust] Generalize Arrow to support MergeSortARROW-10382 - [Rust] Fix typos and spellingARROW-10383 - [Doc] Fix typos and spellingARROW-10384 - [C++] Fix typos and spellingARROW-10385 - [C++][Gandiva] Add support for LLVM 11ARROW-10389 - [Rust][DataFusion] Make the custom source implementation API more explicitARROW-10392 - [C++][Gandiva] Avoid string copy while evaluating IN expressionARROW-10396 - [Rust] [Parquet] Expose SliceableCursor and FileSourceARROW-10398 - [Rust] [Parquet] Re-export parquet::record::api::FieldARROW-10400 - Propagate TLS client peer_identity when using mutual TLSARROW-10402 - [Rust] Improve array equalityARROW-10407 - [C++] Division Support in Decimal256ARROW-10408 - [Java] Upgrade Avro dependency to 1.10ARROW-10410 - [Rust] Some refactoringsARROW-10416 - [R] Support Tables in FlightARROW-10422 - [Rust] Removed unused BinaryArrayBuilderARROW-10424 - [Rust] Simplify code for impl PrimitiveArrayARROW-10428 - [FlightRPC][Java] Add support for HTTP cookiesARROW-10445 - [Rust] Add DoubleEnded to PrimitiveArrayIterARROW-10449 - [Rust] Make dictionary keys be a PrimitiveArrayARROW-10454 - [Rust][Datafusion] support creating ParquetExec from externally resolved file list and schemaARROW-10455 - [Rust] Fix CI cache misses on windowsARROW-10458 - [Rust] [Datafusion] context.create_logical_plan should not take a mutable self referenceARROW-10464 - [Rust] Implement utility to convert TPC-H tbl files to CSV and ParquetARROW-10466 - [Rust] [Website] Update implementation status pageARROW-10467 - [FlightRPC][Java] Ability to pass arbitrary client properties to serverARROW-10468 - [C++][Compute] Refactor FunctionExecutor -> KernelExecutorARROW-10476 - [Rust] Allow string array to be built from iterator of &strARROW-10477 - [Rust] Add support for iterators over binary arraysARROW-10478 - [Dev][Release] Correct Java versions to 3.0.0-SNAPSHOTARROW-10481 - [R] Bindings to add, remove, replace Table columnsARROW-10483 - [C++] Move Executor into a separate headerARROW-10484 - [C++] Future<{void,Status}> could be more genericARROW-10487 - [FlightRPC][C++] Header-based auth in clientsARROW-10490 - [C++][GLib] Fail to build with Xcode 12.0.1ARROW-10492 - [Java][JDBC] Allow users to config the mapping between SQL types and Arrow typesARROW-10504 - [C++] Suppress UBSAN pointer-overflow warning in RapidJSONARROW-10510 - [Rust] [DataFusion] Add benchmarks for COUNT(DISTINCT)ARROW-10515 - [Julia][Doc] Update lists of supported languages to include JuliaARROW-10522 - [R] Allow rename Table and RecordBatch columns with names()ARROW-10526 - [FlightRPC][C++] HTTP cookie handling in clientsARROW-10530 - [R] Optionally use distro package in linuxlibs.RARROW-10531 - [Rust] [DataFusion] Better display for logical plans: Graphviz and Schema informationARROW-10539 - [Packaging][Python] Use GitHub Actions to build wheels for WindowsARROW-10540 - [Rust] Allow unary kernels of arbitrary array typesARROW-10541 - [C++] Add re2 library to core arrow / ARROW_WITH_RE2ARROW-10542 - [C#][Flight] Add beginning on flight code for net coreARROW-10543 - [Developer] Update dev instructions to note there may be a timelagARROW-10552 - [Rust] Remove un-used Result from BufferARROW-10559 - [Rust] [DataFusion] Break up logical_plan/mod.rs into smaller modulesARROW-10561 - [Rust] Simplify `MutableBuffer::write` and `MutableBuffer::write_bytes`ARROW-10562 - [Rust] Potential UB on unsafe codeARROW-10566 - [C++] Array validation should work on ArrayDataARROW-10567 - [C++][FlightRPC] Add options to help increase precision of arrow-flight-benchmarkARROW-10572 - [Rust][DataFusion] Use aHash and std::collections hashmap for aggregates / distinctARROW-10574 - [Python][Parquet] Allow collections for 'in' / 'not in' filter (in addition to sets)ARROW-10575 - [Rust] Rename union.rs to be cosistent with other arraysARROW-10581 - [Doc] IPC dictionary reference to relevant sectionARROW-10582 - [Rust] [DataFusion] Implement "repartition" operatorARROW-10584 - [Rust] [DataFusion] Implement SQL join support using explicit JOIN ON syntaxARROW-10585 - [Rust] [DataFusion] Add join support to DataFrame and LogicalPlanARROW-10586 - [Rust] [DataFusion] Add join support to query plannerARROW-10589 - [Rust]: Implement AVX-512 bit and operationARROW-10590 - [Rust] Remove Date32(Millisecond) from testARROW-10591 - [Rust] Add support to structArrays for MutableArrayDataARROW-10595 - [Rust] Simplify inner loop of min/max kernels for non-null caseARROW-10596 - [Rust] Improve take benchmarkARROW-10598 - [C++] Improve performance of GenerateBitsUnrolledARROW-10604 - [Ruby] Support Decimal256 typeARROW-10607 - [C++][Parquet] Support Reading/Writing Decimal256 type in ParquetARROW-10609 - [Rust] Optimize min/max of non null stringsARROW-10628 - [Rust] Make clippy error on clippy warningsARROW-10633 - [Rust][DataFusion] Dependency version upgradesARROW-10634 - [C#][CI] Change the build version from 2.2 to 3.1 in CIARROW-10636 - [Rust] Remove specialisation from Rust parquetARROW-10637 - [Rust] Add examples to boolean kernelsARROW-10638 - [Rust] Improve tests of boolean kernelsARROW-10639 - [Rust] Simplify signature of is_null and add exampleARROW-10644 - [Python] Consolidate path/filesystem handling in pyarrow.dataset and pyarrow.fsARROW-10646 - [C++][FlightRPC] Disable flaky testARROW-10648 - [Java] Prepare Java codebase for source release without requiring any git tags to be created or pushedARROW-10651 - [C++] alloc-dealloc-mismatch in s3fs.ccARROW-10652 - [C++][Gandiva] Make gandiva cache size configurableARROW-10653 - [Rust]: Update toolchain version to bring new featuresARROW-10654 - [Rust] Specialize parsing of floats / boolsARROW-10660 - [Rust] Implement AVX-512 bit or operationARROW-10665 - [Rust] Add fast paths for common utf8 like patternsARROW-10666 - [Rust] [DataFusion] Support nested SELECT statementsARROW-10669 - [C++][Compute] Support Scalar inputs to boolean kernelsARROW-10672 - [Rust] [DataFusion] Make limit be computed as a streamARROW-10673 - [Rust] [DataFusion] Make sort be computed on the streamARROW-10674 - [Rust] Add integration tests for Decimal typeARROW-10677 - [Rust] Fix Bug and Add tests as documentation showing supported csv parsingARROW-10679 - [Rust] [DataFusion] Implement SQL CASE WHEN physical expressionARROW-10680 - [Rust] [DataFusion] Implement TPC-H Query 12ARROW-10682 - [Rust] Sort kernel performance tuningARROW-10685 - [Rust] [DataFusion] Add support for join on filter pushdown optimizerARROW-10688 - [Rust] [DataFusion] Support CASE WHEN from DataFrame APIARROW-10689 - [Rust] [DataFusion] Support CASE WHEN from SQLARROW-10693 - [Rust] [DataFusion] Add support for the left joinARROW-10696 - [C++] Investigate a bit run reader that would only return runs of set bitsARROW-10697 - [C++] Consolidate bitmap word readersARROW-10703 - [Rust] [DataFusion] Make join not collect left on every partARROW-10704 - [Rust][DataFusion] Remove Nested from expression enumARROW-10708 - [Packaging][deb] Add support for Ubuntu 20.10ARROW-10709 - [Python] Difficult to make an efficient zero-copy file reader in PythonARROW-10712 - [Rust] [DataFusion] Add tests to TPC-H benchmarksARROW-10717 - [Rust] [DataFusion] Add support for right joinARROW-10720 - [C++] Add BasicDecimal256 Rescale SupportARROW-10721 - [C#][CI] Use .NET 3.1 by defaultARROW-10722 - [Rust][DataFusion] Reduce overhead in data types in aggregations / joins, improve benchmarksARROW-10723 - [Packaging][deb][RPM] Enable Parquet encriptionARROW-10724 - [Developer Tools] Add labeler to when PRs need rebaseARROW-10725 - [Python][Compute] Exposing bindings for sort optionsARROW-10728 - [Rust] [DataFusion] Add SQL support for JOIN with USING clauseARROW-10729 - [Rust] [DataFusion] Add SQL support for JOIN using implicit syntaxARROW-10732 - [Rust] [DataFusion] Add SQL support for table/relation aliases and compound identifiersARROW-10733 - [R] Improvements to Linux installation troubleshootingARROW-10740 - [Rust][DataFusion] Remove redundant clones found by clippyARROW-10741 - Apply clippy lints to source code, remove them from ignore listARROW-10742 - [Python] Mask not checked when creating array from numpy arrayARROW-10745 - [Rust] Allocate padding bytes in filter contextARROW-10747 - [Rust] Optimizations for csv readerARROW-10750 - [Rust] [DataFusion] Add SQL support for LEFT and RIGHT joinARROW-10752 - [GLib] Add garrow_schema_has_metadata()ARROW-10754 - [GLib] Add support for metadata to GArrowFieldARROW-10755 - [Rust] [Parquet] Add support for writing boolean typeARROW-10756 - [Rust] Clippy - fix reduntant cloneARROW-10759 - [Rust][DataFusion] Implement support for casting string to date in sql expressionsARROW-10763 - [Rust] Speed up take kernelsARROW-10765 - [Rust] Optimize take strings for non-null arraysARROW-10767 - [Rust] Speed up sum kernel with nullsARROW-10770 - [Rust] Support reading nested JSON listsARROW-10772 - [Rust] Improve take performanceARROW-10775 - [Rust][DataFusion] Use ahash in hash joinARROW-10776 - [C++] Provide iterator access to primitive elements inside an ArrayARROW-10781 - [Rust] [DataFusion] TableProvider should provide row count statisticsARROW-10783 - [Rust] [DataFusion] Implement row count statistics for Parquet TableProviderARROW-10785 - Further optimize take stringARROW-10786 - [Packaging][RPM] Drop support for CentOS 6ARROW-10788 - [C++] Make S3 recursive walks parallelARROW-10789 - [Rust][DataFusion] Make TableProvider dynamically typedARROW-10790 - [C++][Compute] Investigate ChunkedArray sort performanceARROW-10792 - [Rust] [CI] Modulararize CI for faster and smaller buildsARROW-10795 - [Rust] Fix specialization for arrow datatypesARROW-10796 - [C++] Investigate RecordBatch sort performanceARROW-10800 - [Rust] [Parquet] Provide access to the elements of parquet::record::{List, Map}ARROW-10802 - [C++] Remove Dictionary[NullType] special casing in parquet column writerARROW-10808 - [Rust] [DataFusion] Support nested expressions in aggregationsARROW-10809 - [C++] Use Datum for SortIndices() inputARROW-10812 - [Rust] Make BooleanArray not a PrimitiveArrayARROW-10813 - [Rust] [DataFusion] Implement DFSchemaARROW-10814 - [Packaging][deb] Drop support for Debian GNU/Linux StretchARROW-10817 - [Rust] [DataFusion] Implement TypedStringARROW-10820 - [Rust] [DataFusion] Complete TPC-H Benchmark QueriesARROW-10821 - [Rust] [Datafusion] implement negative expressionARROW-10822 - [Rust] [Datafusion] support compiling datafusion with simd supportARROW-10824 - [Rust] Added PartialEq for NullArrayARROW-10825 - [Rust] Add support to NullArrays for MutableArrayDataARROW-10826 - [Rust] Add support for FixedSizeBinary to MutableArrayDataARROW-10827 - [Rust] Extend concatenate to all typesARROW-10828 - [Rust][DataFusion] Enable more clippy lintsARROW-10829 - [Rust] [DataFusion] Implement Into for DFSchema
ARROW-10832 - [Rust] Evaluate latest snapshot flatcARROW-10836 - [Rust] Extend take kernel to FixedSizeListArrayARROW-10838 - [Rust] [CI] Add CI for wasm32 targetARROW-10839 - [Rust] [DataFusion] Implement BETWEEN OperatorARROW-10843 - [C++] Add support for temporal types in sort family kernelsARROW-10845 - [Python][CI] Add python CI build using numpy nightlyARROW-10849 - [Python] Handle numpy deprecation warnings for builtin type aliasesARROW-10851 - [C++] Reduce code size of vector_sort.ccARROW-10857 - [Packaging] Follow PowerTools repository name change on CentOS 8ARROW-10858 - [C++][MSVC] Add missing Boost dependencyARROW-10861 - [Python] Update minimal NumPy version to 1.16.6ARROW-10864 - [Rust] Use standard ordering for floatsARROW-10865 - [Rust][DataFusion] More ergonomic conversion between Schema, SchemaRef, DFSchema, and DFSchemaRefARROW-10867 - build failure on aarch64 with -DARROW_PYTHON=ON and gccARROW-10869 - [GLib] Add garrow_*_sort_indices() and related optionsARROW-10870 - [Julia] Update website with Julia implementationARROW-10871 - [Julia] Setup Julia CI via GitHub ActionsARROW-10873 - [C++] Apple Silicon is reported as arm64 in CMakeARROW-10874 - [Rust][DataFusion] Add table statistics for MemTableARROW-10877 - [Rust] [DataFusion] Add benchmark based on kaggle moviesARROW-10878 - [Rust] Simplify extend_from_sliceARROW-10879 - [Packaging][deb] Restore Debian GNU/Linux Buster configurationARROW-10881 - [C++] EXC_BAD_ACCESS in BaseSetBitRunReader::NextRun
ARROW-10885 - [Rust][DataFusion] Optimize join build vs probe based on statistics on row numberARROW-10887 - [C++][Doc] Document IPC APIARROW-10889 - [Rust] Document our approach to unsafe code in READMEARROW-10890 - [Rust] [DataFusion] JOIN supportARROW-10891 - [Rust][DataFusion] More clippy lintsARROW-10893 - [Rust] [DataFusion] Easier clippy fixesARROW-10896 - [C++][CMake] Rename internal RE2 package name to "re2" from "RE2"ARROW-10900 - [Rust][DataFusion] Resolve TableScan provider eagerlyARROW-10904 - [Python] Add support for Python 3.9 macOS wheelsARROW-10905 - [Python] Add support for Python 3.9 windows wheelsARROW-10908 - [Rust] [DataFusion] Update relevant tpch-queries with BETWEENARROW-10917 - [Rust][Doc] Update feature matrixARROW-10918 - [C++][Doc] Document supported Parquet featuresARROW-10927 - Add Decimal to ArrayBuilderReader for physical type fixed size binaryARROW-10927 - Add Decimal to ArrayBuilderReader for physical type fixed size binaryARROW-10927 - Add Decimal to ArrayBuilderReader for physical type fixed size binaryARROW-10929 - [Rust] Migrate CI tests to stable rustARROW-10933 - [Rust] Update docs in regard to stable rustARROW-10934 - [Python] Tests are failed with fsspec-0.8.5ARROW-10938 - [Rust] upgrade dependency "flatbuffers" to 0.8ARROW-10940 - [Rust] Extend sort kernel to ListArrayARROW-10941 - [Doc][C++] Document supported Parquet encryption featuresARROW-10944 - [Rust] Implement min/max kernels for BooleanArrayARROW-10946 - [Rust] Make ChunkIter not depend on a bufferARROW-10947 - [Rust][DataFusion] Refactor UTF8 to Date32 for PerformanceARROW-10948 - [C++] Always use GTestConfig.cmakeARROW-10949 - [Rust] Avoid clones in getting values of boolean arraysARROW-10951 - [Python][CI] Nightly pandas builds failing because of pytest monkeypatch issueARROW-10952 - [Rust] Add pre-commit hookARROW-10966 - [C++] Use FnOnce for ThreadPool's tasks instead of std::functionARROW-10968 - [Rust][DataFusion] Don't build hash table for right side of the joinARROW-10969 - [Rust][DataFusion] Implement ANSI SQL FunctionsARROW-10985 - [Rust] Update unsafe guidelines for adding JIRA referencesARROW-10986 - [Rust][DataFusion] Add average statistic to TCP-H benchmark tooARROW-10988 - [C++] Require CMake 3.5 or laterARROW-10989 - [Rust] Use slices for iterating primitive arraysARROW-10993 - [CI][macOS] Fix Python 3.9 installation by HomebrewARROW-10995 - [Rust] [DataFusion] Improve parallelism when reading Parquet filesARROW-11004 - [FlightRPC][Python] Header-based auth in clientsARROW-11005 - [Rust] Remove indirection from take kernel and simplify interfaceARROW-11008 - [Rust][DataFusion] Simplify count accumulatorARROW-11009 - [Python] Add environment variable to elect default usage of system memory allocator instead of jemalloc/mimallocARROW-11010 - [Python] `np.float` deprecation warning in `_pandas_logical_type_map`ARROW-11012 - [Rust] [DataFusion] Make write_csv and write_parquet concurrentARROW-11015 - [CI][Gandiva] Move gandiva nightly build from travis to github actionARROW-11018 - [Rust][DataFusion] Add null count column statisticsARROW-11026 - [Rust]: Run tests without requiring environment variablesARROW-11028 - [Rust] Somewhat pedantic pattern-matchesARROW-11029 - [Rust] [DataFusion] Document why join order optimization does not work with filter pushdownARROW-11032 - [C++][FlightRPC] Add benchmark for local RPC through unix socketARROW-11033 - [Rust] CSV writer performance improvementsARROW-11034 - [Rust] rustfmt cleanupARROW-11035 - [Rust] Improve performance of cast to utf8 via FromIterARROW-11037 - [Rust] Improve performance of string fromIterARROW-11038 - [Rust] Remove `BufferBuilderTrait` and associated Result requirement.ARROW-11039 - [Rust] Improve performance for utf8 to float castARROW-11040 - [Rust] Simplify builders with genericsARROW-11042 - [Rust][DataFusion] Increase default batch sizeARROW-11043 - [C++] Add "is_nan" kernelARROW-11046 - [Rust][DataFusion] Add count_distinct to dataframe APIARROW-11049 - [Python] Expose alternate memory poolsARROW-11052 - [Rust] [DataFusion] Implement metrics in join operatorARROW-11053 - [Rust] [DataFusion] Optimize joins with dynamic capacity for output batchesARROW-11054 - Update SQLParser to 0.70ARROW-11055 - [Rust] [DataFusion] Support date_trunc functionARROW-11058 - [Rust] [DataFusion] Implement "coalesce batches" operatorARROW-11063 - [Rust] Validate null counts when building arraysARROW-11064 - [Rust][DataFusion] Speed up hash join on smaller batchesARROW-11072 - [Rust] [Parquet] Support int32 and int64 physical typesARROW-11076 - [Rust][DataFusion] Refactor usage of right indices in hash joinARROW-11079 - [R] Catch up on changelog since 2.0ARROW-11080 - [C++][Dataset] Improvements to implicit castingARROW-11082 - [Rust] Add FFI for LargeUtf8ARROW-11086 - [Rust] Extend take to support more index typesARROW-11091 - [Rust][DataFusion] Fix clippy warning in rust 1.49ARROW-11095 - [Python] Access pyarrow.RecordBatch column by nameARROW-11096 - [Rust] Add FFI for [Large]BinaryARROW-11097 - [Rust] Simplify testsARROW-11099 - [Rust]: Remove unsafe value_slice method from PrimitiveArray and BooleanArrayARROW-11100 - [Rust] Speed up numeric to string cast using lexical_coreARROW-11101 - [Rust] enable "cargo +nightly fmt" in git pre-commit hookARROW-11104 - [GLib] Add append_null/append_nulls to GArrowArrayBuilder and use themARROW-11105 - [Rust] Favor From/Into traits in MutableBufferARROW-11109 - [GLib] Add garrow_array_builder_append_empty_value() and values()ARROW-11110 - [Rust] [Datafusion] context.table should not take a mutable self referenceARROW-11111 - [GLib] Add GArrowFixedSizeBinaryArrayBuilderARROW-11121 - [Developer] Use pull_request_target for PR JIRA integrationARROW-11122 - [Rust] Add FFI for date and timeARROW-11124 - [Doc] Update status matrix for Decimal256ARROW-11125 - [Rust] Implement logical equality for list arraysARROW-11126 - [Rust] Document and test ARROW-10656ARROW-11127 - [C++] Unused cpu_info on non-x86 architectureARROW-11129 - [Rust][DataFusion] Use tokio thread pool for loading parquetARROW-11130 - [Website][CentOS 8][RHEL 8] Enable all required repositories by defaultARROW-11131 - [Rust] Improve performance of bool_equalARROW-11136 - [R] Bindings for is.nanARROW-11137 - [Rust][DataFusion] Fix Clippy needless_range_loop, needless_lifetimesARROW-11138 - [Rust] [DataFusion] Support ltrim, rtrimARROW-11139 - [GLib] Add support for extension typeARROW-11155 - [C++][Packaging] Move gandiva crossbow jobs off of Travis-CIARROW-11158 - [Julia] Implement Decimal256 supportARROW-11159 - [Developer] Consolidate pull request related jobsARROW-11165 - [Rust] [DataFusion] Document the desired SQL dialect for DataFusionARROW-11168 - [Rust] Fix cargo doc warningsARROW-11169 - [Rust] Add a comment explaining where float total_order algorithm came fromARROW-11175 - [R] Small docs fixesARROW-11176 - [R] Expose memory pool name and document setting itARROW-11187 - [Rust] [Parquet] Pin specific parquet-format-rs versionARROW-11188 - [Rust] Implement crypto functions from PostgreSQL dialectARROW-11193 - [Documentation] Add docs for Java ListVectorARROW-11194 - [Rust] Enable SIMD for aarch64ARROW-11195 - [Rust] [DataFusion] Built-in table providers should expose relevant fieldsARROW-11196 - [GLib] Add support for mock, HDFS and S3 file systems with factory functionARROW-11198 - [Packaging][Python] Ensure setuptools version during build supports markdownARROW-11200 - [Rust] [DateFusion] Physical operators and expressions should have public accessor methodsARROW-11201 - [Rust] create_batch_empty - support more typesARROW-11203 - [Developer][Website] Enable JIRA and pull request integrationARROW-11204 - [C++] Fix build failure with bundled gRPC and ProtobufARROW-11205 - [GLib][Dataset] Add GADFileFormat and its familyARROW-11209 - [Rust] DF - Provide better error message on unsupported GROUP BYARROW-11210 - [CI] Restore workflows that had been blocked by INFRAARROW-11212 - [Packaging][Python] Use vcpkg as dependency source for manylinux and windows wheelsARROW-11213 - [Packaging][Python] Dockerize wheel building on windowsARROW-11215 - [CI] Use named volumes by default for caching in docker-composeARROW-11218 - [R] Make SubTreeFileSystem print method more informativeARROW-11219 - [CI][Ruby][MinGW] Reduce CI timeARROW-11221 - [Rust] DF Implement GROUP BY support for Float32/Float64ARROW-11231 - [Packaging] Add mimalloc to Linux buildsARROW-11234 - [CI][Ruby][macOS] Reduce CI timeARROW-11236 - [Java] Bump Jackson to 2.11.4ARROW-11240 - [Packaging][R] Add mimalloc to R packagingARROW-11242 - [CI] Remove CMake 3.2 jobARROW-11245 - [C++][Gandiva] Add support for LLVM 11.1ARROW-11247 - [C++] Infer date32 columns in CSVARROW-11256 - [Packaging][Linux] Don't buffer packaging outputARROW-11272 - [Release][wheel] Remove unsupported Python 3.5 and manylinux1ARROW-11273 - [Release][deb] Remove unsupported Debian GNU/Linux stretchARROW-11278 - [Release][NodeJS] Don't touch ~/.bash_profileARROW-11280 - [Release][APT] Fix minimal build example checkARROW-11281 - [C++] Remove needless runtime RapidJSON dependencyARROW-11282 - [Packaging][deb] Add missing libgflags-dev dependencyARROW-11285 - [Release][APT] Add support for Ubuntu GroovyARROW-11292 - [Release][JS] Use Node.JS LTSARROW-11293 - [C++] Don't require Boost and gflags with find_package(Arrow)ARROW-11307 - [Release][Ubuntu][20.10] Add workaround for dependency issuePARQUET-1566 - [C++] Indicate if null count, distinct count are present in column statisticsBug Fixes
ARROW-2616 - [Python] Cross-compiling PyarrowARROW-6582 - [R] Arrow to R fails with embedded nuls in stringsARROW-7363 - [Python] Add combine_chunks method to ChunkedArrayARROW-7909 - [Website] Add how to install on Red Hat Enterprise LinuxARROW-8258 - [Rust] [Parquet] ArrowReader fails on some timestamp typesARROW-9027 - [Python] Split in multiple files + clean-up pyarrow.parquet testsARROW-9479 - [JS] Table.from fails for zero-item Lists, FixedSizeLists, Maps. ditto Table.emptyARROW-9636 - [Python] Update documentation about 'LZO' compression in parquet.write_tableARROW-9776 - [R] read_feather causes segfault in R if file doesn't existARROW-9897 - [C++][Gandiva] Add to_date() function from patternARROW-9897 - [C++][Gandiva] Add to_date() function from patternARROW-9898 - [C++][Gandiva] Error handling in castINT fails in some enviromentsARROW-9903 - [R] open_dataset freezes opening feather files on WindowsARROW-9963 - [Python] Recognize datetime.timezone.utc as UTC on conversion python->pyarrowARROW-10039 - [Rust] Do not require memory alignment of buffersARROW-10042 - [Rust] Buffer equalities may be incorrectARROW-10080 - [R] Arrow does not release unused memoryARROW-10122 - [Python] Selecting one column of multi-index results in a duplicated value column.ARROW-10145 - [C++][Dataset] Assert integer overflow in partitioning falls back to stringARROW-10146 - [Python] Parquet metadata to_dict raises attribute errorARROW-10174 - [Java] Reading of Dictionary encoded struct vector failsARROW-10177 - [CI][Gandiva] Nightly gandiva-jar-xenial failsARROW-10186 - [Rust] Tests fail when following instructions in READMEARROW-10247 - [C++][Dataset] Cannot write dataset with dictionary column as partition fieldARROW-10264 - [C++][Python] Parquet test failing with HadoopFileSystem URIARROW-10270 - [R] Fix CSV timestamp_parsers test on R-develARROW-10283 - [Python] Python deprecation warning for "PY_SSIZE_T_CLEAN will be required for '#' formats"ARROW-10293 - [Rust] [DataFusion] Fix benchmarksARROW-10294 - [Java] Resolve problems of DecimalVector APIs on ArrowBufsARROW-10321 - [C++] Building AVX512 code when we should notARROW-10333 - [Java] Remove split packages in arrow-memory-core and arrow-vectorsARROW-10345 - [C++] NaN breaks sortingARROW-10346 - [Python] Default S3 region is eu-central-1 even with LANG=CARROW-10348 - [C++] Fix crash on invalid Parquet file (OSS-Fuzz)ARROW-10350 - [Rust] parquet_derive crate cannot be published to crates.ioARROW-10353 - [C++] Parquet decompresses DataPageV2 pages even if is_compressed==0ARROW-10358 - [R] Followups to 2.0.0 releaseARROW-10365 - [R] Remove duplicate setting of S3 flag on macOSARROW-10369 - [Dev] Fix archery release utility test casesARROW-10371 - [R] Linux system requirements check needs to support older cmake versionsARROW-10386 - [R] List column class attributes not preserved in roundtripARROW-10388 - [Java] Fix Spark integration build failureARROW-10390 - [Rust] [Parquet] Regression Can not implement custom ParquetWriter because `TryClone` is not publically exportedARROW-10393 - [Rust]: Fix null value reading in jsonreader for both dictionary and stringbuildersARROW-10394 - [Rust] [Large]BinaryArray can be created from non-binary datatypesARROW-10397 - [C++] Outdated and confusing comment on dictionary indicesARROW-10399 - [R] Fix performance regression from cpp11::r_stringARROW-10411 - [C++] Fix incorrect child array lengths for Concatenate of FixedSizeListARROW-10412 - [C++] CMake Build Fails with grpc 1.33.1, "GRPC_CPP_PLUGIN-NOTFOUND: program not found or is not executable"ARROW-10413 - [Rust] [Parquet] Unignore some roundtrip tests that are passing nowARROW-10414 - [R] open_dataset doesn't work with absolute/expanded paths on WindowsARROW-10426 - [C++] Arrow type large_string cannot be written to Parquet type column descriptorARROW-10433 - [Python] pyarrow doesn't work with s3fs>=0.5ARROW-10434 - [Rust] Debug formatting arrays with lengths greater than 10 and less than 20 produces incorrect valuesARROW-10441 - [FlightRPC][Java] FlightClients from FlightGrpcUtils#createFlightClient shutdown gRPC channel when closedARROW-10446 - [C++][Python] Timezone aware pd.Timestamp's are incorrectly converted to Timestamp arrysARROW-10448 - [Rust] PrimitiveArray::new can create arrays not in specARROW-10453 - [Rust] [DataFusion] Performance degredation after removing specializationARROW-10457 - [CI] Fix Spark branch-3.0 integration testsARROW-10461 - [Rust] Offset related bug in BitChunks::remainder_bitsARROW-10462 - [Python] ParquetDatasetPiece's path broken when using fsspec fs on WindowsARROW-10463 - [R] Better messaging for currently unsupported CSV options in open_datasetARROW-10470 - [R] Fix missing file error causing NYC taxi example to failARROW-10471 - [CI][Python] Ensure we have tests with s3fs and run those on CIARROW-10472 - [C++][Python] casting a scalar timestamp to date32 results in Aborted (core dump)ARROW-10475 - [С++][FlightRPC] Arrow Flight Server / Client cannot be initialized with Ipv6 hostARROW-10480 - [Python] Parquet write_table creates gzipped Parquet file, not Parquet with gzip compressionARROW-10482 - [Python] Specifying compression type on a column basis when writing Parquet not workingARROW-10489 - [C++] Unable to configure or make with intel compilerARROW-10491 - [FlightRPC][Java] Fix NPE when using FlightProducer without interceptorsARROW-10493 - [C++][Parquet] Writing nullable nested strings results in wrong data in fileARROW-10495 - [C++] find_package(Arrow) is broken on Ubuntu 18ARROW-10496 - [R][CI] Fix conda-r jobARROW-10499 - [C++][Java] Fix ORC Java JNI CrashARROW-10502 - [C++/Python] CUDA detection messes up nightly conda-win buildsARROW-10503 - [C++] Uriparser will not compile using Intel compilerARROW-10508 - [Java] Allow FixedSizeListVector to have empty childrenARROW-10509 - [C++] Define operator<<(ostream, ParquetException) for clang+WindowsARROW-10511 - [Python] Table.to_pandas() failing when timezone-awareness mismatch in metadataARROW-10518 - Fix cast function issues in gandivaARROW-10519 - [Python] Deadlock when PyArrow imports Pandas from multiple threadsARROW-10525 - [C++] Fix crash on unsupported IPC stream (OSS-Fuzz)ARROW-10532 - [Python] Mangled pandas_metadata when specified schema has different order as DataFrame columnsARROW-10545 - [C++] Fix crash on invalid Parquet file (OSS-Fuzz)ARROW-10546 - [Python] Deprecate the S3FSWrapper classARROW-10547 - [Rust][DataFusion] Filter pushdown loses filters if below a user defined nodeARROW-10551 - [Rust]: Fix unreproducible benchmarksARROW-10558 - [Python] Filesystem S3 tests not independent (native s3 influences s3fs)ARROW-10560 - [Python] Crash when creating array with string over 2GBARROW-10563 - [Packaging][C++] CMake find_package(Arrow 2.0 CONFIG REQUIRED) brokenARROW-10565 - [Python] Table.from_batches and Table.from_pandas have argument Schema_schema in documentation instead of schemaARROW-10568 - [C++][Parquet] Parquet writer crashes process when Tell() does not succeedARROW-10569 - [C++][Python] Poor Table filtering performanceARROW-10577 - [Rust][DataFusion] Hash Aggregator stream finishes unexpectedly after going to Pending stateARROW-10578 - [C++] Comparison kernels crashing for string array with null string scalarARROW-10610 - [C++] arrow-utility-test and arrow-csv-test causes failures on a big-endian platformARROW-10616 - [Developer] Expand PR labeler to all supported languagesARROW-10617 - [Python] RecordBatchStreamReader's iterator doesn't work with python 3.8ARROW-10619 - [C++] Fix crash on unsupported IPC stream (OSS-Fuzz)ARROW-10620 - [Rust][Parquet] move column chunk range logic to metadata.rsARROW-10621 - [Java] flight-cpre test causes a failure on s390xARROW-10622 - [R] Nameof<>() is incorrect in r-arrow build environmentARROW-10623 - [R] Version 1.0.1 breaks data.frame attributes when reading file written by 2.0.0ARROW-10624 - [R] Proactively remove "problems" attributesARROW-10627 - [Rust] Github master does not compile for WASM targetARROW-10629 - [CI] MinGW builds broken on Github ActionsARROW-10631 - [Rust] Equality of fixed-sized binary is incorrect.ARROW-10642 - [R] Can't get Table from RecordBatchReader with 0 batchesARROW-10656 - [Rust] New RecordBatch requires exact match of Data TypesARROW-10656 - [Rust] New RecordBatch requires exact match of Data TypesARROW-10661 - [C#] Fix benchmarking projectARROW-10662 - [Java] Avoid integer overflow for Json file readerARROW-10663 - [C++/Doc] The IsIn kernel ignores the skip_nulls option of SetLookupOptionsARROW-10667 - [Rust] [Parquet] Add a convenience type for writing Parquet to memoryARROW-10668 - [R] Filtering does not work with .data pronounARROW-10681 - [Rust] [DataFusion] TPC-H Query 12 fails with scheduler errorARROW-10684 - [Rust] Logical equality should consider parent array nullabilityARROW-10690 - [Java] ComplexCopier gives incorrect result for list vector if target vector is non-emptyARROW-10692 - [Rust] Segfault while array buffer appendARROW-10699 - [C++] BitmapUInt64Reader doesn't work on big-endianARROW-10701 - [Rust] [Datafusion] Benchmark sort_limit_query_sql fails because order by clause specifies column index instead of expressionARROW-10705 - [Rust] Lifetime annotations in the IPC writer are too strict, preventing code reuseARROW-10710 - [Rust] Example flight server is broken after tokio upgrade (among other things)ARROW-10711 - [CI] Remove set-env from auto-tune to work with new GHA settingsARROW-10719 - [C#] ArrowStreamWriter doesn't write schema metadataARROW-10746 - [C++] Use GTEST_SKIP in parquet encoding testsARROW-10748 - [Java] TimeStampMilliVector cannot be cast to TimeStampMilliTZVectorARROW-10749 - [C++] Incorrect string format for Datum with the collection typeARROW-10751 - [C++] Add RE2 to minimal build exampleARROW-10753 - [Rust] [DataFusion] Negative numbers in SQL WHERE clause not parsed correctlyARROW-10757 - [Rust] [CI] Sporadic failures due to disk filling upARROW-10760 - [Rust] [DataFusion] Predicate push down does not support joins correctlyARROW-10769 - [CI] Integration tests are failing in masterARROW-10774 - [R] Set minimum cpp11 versionARROW-10777 - [Packaging][Python] PyPI pyarrow source dist (sdist) contains architecture dependent binariesARROW-10778 - [Python] RowGroupInfo.statistics errors for empty row groupARROW-10779 - [Java] writeNull method in UnionListWriter doesn't work correctly if validity at that index is already setARROW-10780 - [R] Update known R installation issues for CentOS 7ARROW-10791 - [Rust] StreamReader, read_dictionary duplicating schema infoARROW-10801 - [Rust] [Flight] Support sending FlightData for Dictionaries with that of a RecordBatchARROW-10803 - [R] Support R >= 3.3 and add CIARROW-10804 - [Rust] Remove UB on parquet crateARROW-10807 - [Rust][DataFusion] Avoid double hashingARROW-10810 - [Rust] Speed up comparison kernelsARROW-10811 - [R][CI] Remove nightly centos6 buildARROW-10823 - MutableArrayData with use_null false yields wrong resultsARROW-10830 - [Rust] json reader should not hard crash on invalid jsonARROW-10833 - [Python] Avoid usage of NumPy's PyArray_DescrCheck macroARROW-10834 - [R] Fix print method for SubTreeFileSystemARROW-10837 - [Rust] Use `Vec` for hash key instead
ARROW-10840 - [C++] Parquet FileMetaData does not have key_value_metadata when built from FileMetaDataBuilderARROW-10842 - [Rust] decouple IO from json schema inference codeARROW-10844 - [Rust] [DataFusion] join of two DataFrames is not possibleARROW-10850 - [R] Unrecognized compression type: LZ4ARROW-10852 - [C++] AssertTablesEqual(verbose=true) segfaults if the left array has more rowsARROW-10854 - [Rust] [DataFusion] Simplified logical scansARROW-10855 - [Python][Numpy] ArrowTypeError after upgrading NumPy to 1.20.0rc1ARROW-10856 - [R] CentOS 7 not correctly identifying compiler versionARROW-10859 - [Rust] [DataFusion] Make collect not require ExecutionContextARROW-10860 - [Java] Avoid integer overflow for generated classes in VectorARROW-10863 - [Python] ExtensionArray.to_pandas not workingARROW-10863 - [Python] ExtensionArray.to_pandas not workingARROW-10875 - simplify simd cfg checkARROW-10876 - [Rust] json reader should validate value typeARROW-10897 - [Rust] Replace Arc by String in DataType::Timestamp
ARROW-10907 - [Rust] Cast UTF8 to Date64 IncorrectARROW-10913 - [Python][Doc] Code block typo in filesystems docsARROW-10914 - [Rust]: SIMD implementation of arithmetic kernels reads out of boundsARROW-10915 - [Rust] Make ARROW_TEST_DATA and PARQUET_TEST_DATA absolute dirsARROW-10921 - `TypeError: 'coroutine' object is not iterable` when reading parquet partitions via s3fs >= 0.5 with pyarrowARROW-10930 - [Python] LargeListType doesn't have a value_fieldARROW-10932 - [C++] BinaryMemoTable::CopyOffsets access out-of-bound address when data is emptyARROW-10932 - [C++] BinaryMemoTable::CopyOffsets access out-of-bound address when data is emptyARROW-10942 - [C++] S3FileSystem::Impl::IsEmptyDirectory fails on Amazon S3ARROW-10943 - [Rust] Intermittent build failure in parquet encodingARROW-10954 - [C++][Doc] PlasmaClient is threadSafe now, doc not updateARROW-10955 - [C++] Reading empty json lists results in invalid non-nullable null typeARROW-10960 - [C++][FlightRPC] Missing protobuf data_body should result in default value of empty bytes, not nullARROW-10962 - [Java][FlightRPC] FlightData deserializer should accept missing fieldsARROW-10967 - [Rust] Make env vars ARROW_TEST_DATA and PARQUET_TEST_DATA optionalARROW-10990 - [Rust]: SIMD implementation of compare kernels reads out of boundsARROW-10994 - [Rust] Fix bugs in TPC-H file conversionARROW-10996 - [Rust] Return error messages via Result for get_arrow_schema_from_metadataARROW-10999 - [Rust] TPC-H parquet files cannot be read by Apache SparkARROW-11014 - [Rust] [DataFusion] ParquetExec reports incorrect statisticsARROW-11023 - [C++][CMake] gRPC doesn't respect CMAKE_CXX_COMPILERARROW-11024 - [C++][Parquet] Writing List to parquet sometimes writes wrong data
ARROW-11025 - [Rust] Bench for boolean kernels measure array creationARROW-11030 - [Rust] [DataFusion] HashJoinExec slow with many batchesARROW-11048 - [Rust] Add bench to MutableBufferARROW-11050 - [R] Handle RecordBatch in write_parquetARROW-11067 - [C++] CSV reader returns nulls for some strings on macOSARROW-11069 - [C++] Parquet writer incorrect data being written when data type is structARROW-11073 - [Rust] Lint Error on CI Tests in /arrow/rust/arrow/src/ipc/reader.rsARROW-11083 - [CI] Build "Source Release and Merge Script" is brokenARROW-11084 - [Rust] Clippy failing in masterARROW-11085 - [Rust] Rust CI no longer works b/c it uses action-rs: Migrate CI away from action-rs/*ARROW-11092 - CI move offending workflows to separate filesARROW-11102 - [Rust][DataFusion] fmt::Debug for ScalarValue(Utf8) is always quotedARROW-11113 - [Rust] support as_struct_array castARROW-11114 - [Java] Metadata serialization is broken for Field classARROW-11132 - [CI] Use pip to install crossbow's dependencies for the comment botARROW-11144 - [C++][Python][CI] Fix HDFS nightly buildARROW-11152 - [CI][C++] Fix Homebrew numpy installation on macOS buildsARROW-11162 - [C++] Fix crash on Decimal256 Parquet file (OSS-Fuzz)ARROW-11163 - [C++][Python] Compressed Feather file written with pyarrow 0.17 not readable in pyarrow 2.0.0+ARROW-11166 - [Python][Compute] Add bindings for ProjectOptionsARROW-11171 - [Go] Build fails on s390x with noasm tagARROW-11189 - [Developer] Achery benchmark diff cannot compare two jsonsARROW-11190 - [C++][Dataset] Clean up compiler warningsARROW-11202 - [R][CI] Nightly builds not happening (or artifacts not exported)ARROW-11224 - [R] don't test metadata serialization on old R versionsARROW-11226 - [Python][CI] Filesystem tests failing with s3fs 0.5.2ARROW-11227 - [Python][CI] AMD64 Conda Python 3.7 Pandas 0.24 cron job failing in to_pandas extension dtype testARROW-11229 - [C++][Dataset] Static build is failedARROW-11230 - [R] Fix build failures on Windows when multiple libarrow binaries foundARROW-11232 - [C++] Table::CombineChunks() returns incorrect results if Table has no columnARROW-11233 - [C++][Flight] Fail to link with bundled gRPC and AbseilARROW-11237 - [C++] Compiler error with GLog and unity build enabledARROW-11251 - [CI] Make sure that devtoolset-8 is really installed + being usedARROW-11253 - [R] Make sure that large metadata tests are reproducibleARROW-11255 - [Packaging][Conda][macOS] Fix Python versionARROW-11271 - [Rust] [Parquet] List schema to Arrow parser misinterpreting child nullabilityARROW-11274 - [Packaging][wheel][Windows] Fix wheels path for GemfuryARROW-11275 - [Packaging][wheel][Linux] Fix paths for GemfuryARROW-11283 - [Julia] Fix install linkARROW-11286 - [Release][Yum] Fix minimal build example checkARROW-11287 - [Packaging][RPM] Add missing dependenciesARROW-11301 - [C++] Fix reading LZ4-compressed Parquet files produced by Java Parquet implementationARROW-11302 - [Release][Python] Remove verification of python 3.5 wheel on macOSARROW-11306 - [Packaging][Ubuntu][16.04] Add missing libprotobuf-dev dependencyPARQUET-1935 - [C++][Parquet] nullptr access violation when writing arrays of non-nullable valuesApache Arrow 2.0.0 (2020-10-13)
Bug Fixes
ARROW-2367 - [Python] ListArray has trouble with sizes greater than kMaximumCapacityARROW-4189 - [CI] [Rust] Fix broken cargo coverageARROW-4917 - [C++] orc_ep fails in cpp-alpine dockerARROW-5578 - [C++][Flight] Flight does not build out of the box on Alpine LinuxARROW-7226 - [JSON][Python] Json loader fails on example in documentation.ARROW-7384 - [Website] Fix search indexing warning reported by GoogleARROW-7517 - [C++] Builder does not honour dictionary type provided during initializationARROW-7663 - [Python] from_pandas gives TypeError instead of ArrowTypeError in some casesARROW-7903 - [Rust] [DataFusion] Upgrade SQLParser dependency for DataFusionARROW-7957 - [Python] ParquetDataset cannot take HadoopFileSystem as filesystemARROW-8265 - [Rust] [DataFusion] Table API collect() should not require contextARROW-8394 - [JS] Typescript compiler errors for arrow d.ts files, when using es2015-esm packageARROW-8735 - [Rust] [Parquet] Parquet crate fails to compile on Arm architectureARROW-8749 - [C++] IpcFormatWriter writes dictionary batches with wrong IDARROW-8773 - [Python] pyarrow schema.empty_table() does not preserve nullability of fieldsARROW-9028 - [R] Should be able to convert an empty tableARROW-9096 - [Python] Pandas roundtrip with object-dtype column labels with integer values: data type "integer" not understoodARROW-9177 - [C++][Parquet] Tracking issue for cross-implementation LZ4 Parquet compression compatibilityARROW-9414 - [C++] apt package includes headers for S3 interface, but no supportARROW-9462 - [Go] The Indentation after the first Record arrjson writer is missingARROW-9463 - [Go] The writer is double closed in TestReadWriteARROW-9490 - [Python] pyarrow array creation for specific set of numpy scalars failsARROW-9495 - [C++] Equality assertions don't handle Inf / -Inf properlyARROW-9520 - [Rust] [DataFusion] Can't alias an aggregate expressionARROW-9528 - [Python] Honor tzinfo information when converting from datetime to pyarrowARROW-9532 - [Python] Building pyarrow for MacPorts on macOSARROW-9535 - [Python] Remove symlink fixes from conda recipeARROW-9536 - Missing parameters in PlasmaOutOfMemoryException.javaARROW-9541 - [C++] CMakeLists requires UTF8PROC_STATIC when building static libraryARROW-9544 - [R] version argument of write_parquet not workingARROW-9546 - [Python] Clean up Pandas Metadata Conversion testARROW-9548 - [Go] Test output files in tmp directory are not removed correctlyARROW-9549 - [Rust] Parquet no longer buildsARROW-9554 - [Java] FixedWidthInPlaceVectorSorter sometimes produces wrong resultARROW-9556 - [Python][C++] Segfaults in UnionArray with null valuesARROW-9560 - [Packaging] conda recipes failing due to missing conda-forge.ymlARROW-9569 - [CI][R] Fix rtools35 builds for msys2 key changeARROW-9570 - [Doc] Clean up sphinx sidebarARROW-9573 - [Python] Parquet doesn't load when partitioned column starts with '_'ARROW-9574 - [R] Cleanups for CRAN 1.0.0 releaseARROW-9575 - [R] gcc-UBSAN failure on CRANARROW-9577 - [Python][C++] posix_madvise error on Debian in pyarrow 1.0.0ARROW-9583 - [Rust] Offset is mishandled in arithmetic and boolean compute kernelsARROW-9588 - [C++] clang/win: Copy constructor of ParquetInvalidOrCorruptedFileException not correctly triggeredARROW-9589 - [C++/R] arrow_exports.h contains structs declared as classARROW-9592 - [CI] Update homebrew before calling brew bundleARROW-9596 - [CI][Crossbow] Fix homebrew-cpp again, againARROW-9597 - [C++] AddAlias in compute::FunctionRegistry should be synchronizedARROW-9598 - [C++][Parquet] Spaced definition levels is not assigned correctly.ARROW-9599 - [CI] Appveyor toolchain build fails because CMake detects different C and C++ compilersARROW-9600 - [Rust] When used as a crate dependency, arrow-flight is rebuilt on every invocation of cargo buildARROW-9600 - [Rust] When used as a crate dependency, arrow-flight is rebuilt on every invocation of cargo buildARROW-9602 - [R] Improve cmake detection in Linux buildARROW-9603 - [C++][Parquet] Write Arrow relies on unspecified behavior for nested typesARROW-9606 - [C++][Dataset] in expressions don't work with >1 partition levelsARROW-9609 - [C++] CSV datasets don't materialize virtual columnsARROW-9621 - [Python] test_move_file() is failed with fsspec 0.8.0ARROW-9622 - [Java] ComplexCopier fails if a structvector has a child UnionVector with nullsARROW-9628 - [Rust] Clippy PR test failing intermittently on Rust / AMD64 MacOSARROW-9629 - [Python] Kartothek integration tests failing due to missing freezegun moduleARROW-9631 - [Rust] Arrow crate should not depend on flightARROW-9631 - [Rust] Arrow crate should not depend on flightARROW-9642 - [C++] Let MakeBuilder refer DictionaryType's index_type for deciding the starting bit width of the indicesARROW-9643 - [C++] Illegal instruction on haswell cpuARROW-9644 - [C++][Dataset] Do not check for ignore_prefixes in the base pathARROW-9652 - [Rust][DataFusion] Panic trying to select * from a CSV (panicked at 'index out of bounds: the len is 0 but the index is 0)ARROW-9653 - [Rust][DataFusion] Multi-column Group by: Invalid Argument ErrorARROW-9659 - [C++] RecordBatchStreamReader throws on CUDA device buffersARROW-9660 - [C++] IPC - dictionaries in mapsARROW-9666 - [Python][wheel][Windows] library missing failure by ARROW-9412ARROW-9670 - [C++][FlightRPC] Close()ing a DoPut with an ongoing read locks up the clientARROW-9684 - [C++] Fix undefined behaviour on invalid IPC / Parquet input (OSS-Fuzz)ARROW-9692 - [Python] distutils import warningARROW-9693 - [CI][Docs] Nightly docs build failsARROW-9696 - [Rust] [Datafusion] nested binary expressions brokenARROW-9698 - [C++] Revert "Add -NDEBUG flag to arrow.pc"ARROW-9700 - [Python] create_library_symlinks doesn't work in macosARROW-9712 - [Rust] [DataFusion] ParquetScanExec panics on errorARROW-9714 - [Rust] [DataFusion] TypeCoercionRule not implemented for Limit or SortARROW-9716 - [Rust] [DataFusion] MergeExec should have concurrency limitARROW-9726 - [Rust] [DataFusion] ParquetScanExec launches threads too earlyARROW-9727 - [C++] Fix crash on invalid IPC input (OSS-Fuzz)ARROW-9729 - [Java] Error Prone causes other annotation processors to not work with EclipseARROW-9733 - [Rust][DataFusion] Aggregates COUNT/MIN/MAX don't work on VARCHAR columnsARROW-9734 - [Rust] [DataFusion] TableProvider.scan executing partitions prematurelyARROW-9741 - [Rust] [DataFusion] Incorrect count in TPC-H query 1 result setARROW-9743 - [R] Sanitize paths in open_datasetARROW-9744 - [Python] Failed to install on aarch64ARROW-9764 - [CI][Java] Push wrong Docker imageARROW-9768 - [Python] Pyarrow allows for unsafe conversions of datetime objects to timestamp nanosecondsARROW-9768 - [Python] Pyarrow allows for unsafe conversions of datetime objects to timestamp nanosecondsARROW-9778 - [Rust] [DataFusion] Logical and physical schemas' nullability does not match in 8 out of 20 end-to-end testsARROW-9783 - [Rust] [DataFusion] Logical aggregate expressions require explicit data typeARROW-9785 - [Python] pyarrow/tests/test_fs.py::test_s3_options too slowARROW-9789 - [C++] Don't install jemalloc in parallelARROW-9790 - [Rust] [Parquet] ParquetFileArrowReader fails to decode all pages if batches fall exactly on row group boundariesARROW-9790 - [Rust] [Parquet] ParquetFileArrowReader fails to decode all pages if batches fall exactly on row group boundariesARROW-9793 - [Rust] [DataFusion] Tests failing in masterARROW-9797 - [Rust] AMD64 Conda Integration Tests is failing for the Master branchARROW-9799 - [Rust] [DataFusion] Implementation of physical binary expression get_type method is incorrectARROW-9800 - [Rust] [Parquet] "min" and "max" written to standard out when writing columnsARROW-9809 - [Rust] [DataFusion] logical schema = physical schema is not trueARROW-9814 - [Python] Crash in test_parquet.py::test_read_partitioned_directory_s3fsARROW-9815 - [Rust] [DataFusion] Deadlock in creation of physical plan with two udfsARROW-9815 - [Rust] [DataFusion] Deadlock in creation of physical plan with two udfsARROW-9815 - [Rust] [DataFusion] Deadlock in creation of physical plan with two udfsARROW-9816 - [C++] Escape quotes in config.hARROW-9827 - [Python] pandas.read_parquet fails for wide parquet files and pyarrow 1.0.XARROW-9831 - [Rust] [DataFusion] Fix compilation errorARROW-9840 - [Python] Python fs documentation out of date with codeARROW-9846 - [Rust] Master branch broken buildARROW-9851 - [C++] Valgrind errors due to unrecognized instructionsARROW-9852 - [C++] Fix crash on invalid IPC input (OSS-Fuzz)ARROW-9852 - [C++] Fix crash on invalid IPC input (OSS-Fuzz)ARROW-9855 - [R] Fix bad merge/Rcpp conflictARROW-9859 - [C++] S3 FileSystemFromUri with special char in secret key failsARROW-9864 - [Python] pathlib.Path not supported in write_to_dataset with partition columnsARROW-9874 - [C++] NewStreamWriter / NewFileWriter don't own output streamARROW-9876 - [CI][C++] Travis ARM jobs timeoutARROW-9877 - [C++][CI] homebrew-cpp fails due to avx512ARROW-9879 - [Python] ChunkedArray.__getitem__ doesn't work with numpy scalarsARROW-9882 - [C++/Python] Update conda-forge-pinning to 3 for OSX conda packagesARROW-9883 - [R] Fix linuxlibs.R install script for R < 3.6ARROW-9888 - [Rust] [DataFusion] ExecutionContext can not be shared between threadsARROW-9889 - [Rust][DataFusion] Datafusion CLI: CREATE EXTERNAL TABLE errors with "Unsupported logical plan variant"ARROW-9897 - [C++][Gandiva] Add to_date() function from patternARROW-9906 - [Python] Crash in test_parquet.py::test_parquet_writer_filesystem_s3_uri (closing NativeFile from S3FileSystem)ARROW-9913 - [C++] Outputs of Decimal128::FromString depend on presence of one anotherARROW-9920 - [Python] pyarrow.concat_arrays segfaults when passing it a chunked arrayARROW-9922 - [Rust] Add `try_from(Vec>)` to StructArray
ARROW-9924 - [Python] Performance regression reading individual Parquet files using Dataset interfaceARROW-9931 - [C++] Fix undefined behaviour on invalid IPC (OSS-Fuzz)ARROW-9932 - [R] Arrow 1.0.1 R package fails to install on R3.4 over linuxARROW-9936 - [Python] Fix / test relative file paths in pyarrow.parquetARROW-9937 - [Rust] [DataFusion] Average is not correctARROW-9943 - [C++] Arrow metadata not applied recursively when reading Parquet fileARROW-9946 - [R] ParquetFileWriter segfaults when `sink` is a stringARROW-9953 - [R] Declare minimum version for bit64ARROW-9962 - [Python] Conversion to pandas with index column using fixed timezone failsARROW-9968 - [C++] UBSAN link failure with __int8_tARROW-9969 - [C++] RecordBatchBuilder yields invalid result with dictionary fieldsARROW-9970 - [Go] checkptr failures in sum methodsARROW-9972 - [CI] Work around grpc-re2 clash on HomebrewARROW-9973 - [Java] JDBC DateConsumer does not allow dates before epochARROW-9976 - [Python] ArrowCapacityError when doing Table.from_pandas with large dataframeARROW-9990 - [Rust] [DataFusion] NOT is not plannableARROW-9993 - [Python] Tzinfo - string roundtrip fails on pytz.StaticTzInfo objectsARROW-9994 - [C++][Python] Auto chunking nested array containing binary-like fields result malformed outputARROW-9996 - [C++] Dictionary is unset when calling DictionaryArray.GetScalar for null valuesARROW-10003 - [C++] Create directories in CopyFiles when copying within the same filesystemARROW-10008 - [Python] pyarrow.parquet.read_table fails with predicate pushdown on categorical data with use_legacy_dataset=FalseARROW-10011 - [C++] Make FindRE2.cmake re-entrantARROW-10012 - [C++] Sporadic failures in CopyFiles testARROW-10013 - [C++][CI] Flight test failure in TestFlightClient.GenericOptionsARROW-10017 - [Java] LargeMemoryUtil.checkedCastToInt has buggy logicARROW-10022 - [C++] [Compute] core dumped on some scalar-arithmetic-benchmarkARROW-10027 - [Python] Incorrect null column returned when using a dataset filter expression.ARROW-10034 - [Rust] Master build brokenARROW-10041 - [Rust] Possible to create LargeStringArray with DataType::Utf8ARROW-10047 - [CI] Conda integration tests failing with cmake errorARROW-10048 - [Rust] Error in aggregate of min/max for stringsARROW-10049 - [C++/Python] Sync conda recipe with conda-forgeARROW-10060 - [Rust] [DataFusion] MergeExec currently discards partitions with errorsARROW-10062 - [Rust]: Fix for null elems for DoubleEndedIter for DictArrayARROW-10073 - [Python] Test test_parquet_nested_storage relies on dict item orderingARROW-10081 - [C++/Python] Fix bash syntax in drone.io conda buildsARROW-10085 - [C++] S3 tests fail on AppVeyorARROW-10087 - [CI] Fix nightly docs jobARROW-10098 - [R][Doc] Fix copy_files doc mismatchARROW-10104 - [Python] Separate tests into its own conda packageARROW-10114 - [R] Segfault in to_dataframe_parallel with deeply nested structsARROW-10116 - [Python][Packaging] Fix gRPC linking error in macOS wheels buildsARROW-10119 - [C++] Fix Parquet crashes on invalid input (OSS-Fuzz)ARROW-10121 - [C++][Python] Variable dictionaries do not survive roundtrip to IPC streamARROW-10124 - [R] Write functions don't follow umask settingARROW-10125 - [R] Int64 downcast check doesn't consider all chunksARROW-10130 - [C++][Dataset] ParquetFileFragment::SplitByRowGroup does not preserve "complete_metadata" statusARROW-10136 - [Rust][Arrow] Nulls are transformed into "" after filtering for StringArrayARROW-10137 - [R] Fix cpp helper that breaks if libarrow is not presentARROW-10147 - [Python] Constructing pandas metadata fails if an Index name is not JSON-serializable by defaultARROW-10150 - [C++] Fix crashes on invalid Parquet file (OSS-Fuzz)ARROW-10169 - [Rust] Nulls should be rendered as "" rather than default value when pretty printing arraysARROW-10175 - [CI] Nightly hdfs integration test job failsARROW-10176 - [CI] Nightly valgrind job failsARROW-10178 - [CI] Fix spark master integration test build setupARROW-10179 - [Rust] Labeler is not labelingARROW-10181 - [Rust] Arrow tests fail to compile on Raspberry Pi (32 bit)ARROW-10188 - [Rust] [DataFusion] Some examples are brokenARROW-10189 - [Doc] C data interface example for i32 uses `l`, not `i`, in the formatARROW-10192 - [C++][Python] Segfault when converting nested struct array with dictionary field to pandas seriesARROW-10193 - [Python] Segfault when converting to fixed size binary arrayARROW-10200 - [Java][CI] Fix failure of Java CI on s390xARROW-10204 - [RUST] [Datafusion] Test failure in aggregate_grouped_empty with simd feature enabledARROW-10214 - [Python] UnicodeDecodeError when printing schema with binary metadataARROW-10226 - [Rust] [Parquet] Parquet reader reading wrong columns in some batches within a parquet fileARROW-10230 - [JS][Doc] JavaScript documentation fails to buildARROW-10232 - FixedSizeListArray is incorrectly written/read to/from parquetARROW-10234 - [C++][Gandiva] Fix logic of round() for floats/decimals in GandivaARROW-10237 - [C++] Duplicate values in a dictionary result in corrupted parquetARROW-10238 - [C#] List is broken
ARROW-10239 - [C++] aws-sdk-cpp apparently requires zlib tooARROW-10244 - [Python][Docs] Add docs on using pyarrow.dataset.parquet_datasetARROW-10248 - [C++][Dataset] Dataset writing does not write schema metadataARROW-10262 - [C++] Some TypeClass in Scalar classes seem incorrectARROW-10271 - [Rust] packed_simd is broken and continued under a new projectARROW-10279 - [Release][Python] Fix verification script to align with the new macos wheel platform tagsARROW-10280 - [Packaging][Python] Fix macOS wheel artifact patternsARROW-10281 - [Python] Fix warnings when running testsARROW-10284 - [Python] Pyarrow is raising deprecation warning about filesystems on importARROW-10285 - [Python] pyarrow.orc submodule is using deprecated functionalityARROW-10286 - [C++][Flight] Misleading CMake errorsARROW-10288 - [C++] Compilation fails on i386ARROW-10290 - [C++] List POP_BACK is not available in older CMake versionsNew Features and Improvements
ARROW-983 - [C++] Implement InputStream and OutputStream classes for interacting with socket connectionsARROW-1105 - [C++] SQLite record batch readerARROW-1509 - [Python] Write serialized object as a stream of encapsulated IPC messagesARROW-1669 - [C++] Consider adding Abseil (Google C++11 standard library extensions) to toolchainARROW-1797 - [C++] Implement binary arithmetic kernels for numeric arraysARROW-2164 - [C++] Clean up unnecessary decimal module refsARROW-3080 - [Python] Unify Arrow to Python object conversion pathsARROW-3757 - [R] R bindings for Flight RPC clientARROW-3872 - [R] Add ad hoc test of feather compatibilityARROW-4046 - [Python/CI] Exercise large memory testsARROW-4248 - [C++][Plasma] Build on Windows / Visual StudioARROW-4685 - [C++] Update Boost to 1.69 in manylinux1 docker imageARROW-4927 - [Rust] Update top level README to describe current functionalityARROW-4957 - [Rust] [DataFusion] Implement get_supertype correctlyARROW-4965 - [Python] Timestamp array type detection should use tzname of datetime.datetime objectsARROW-5034 - [C#] ArrowStreamWriter should expose synchronous Write methodsARROW-5123 - [Rust] derive RecordWriter from struct definitionsARROW-6075 - [FlightRPC] Handle uncaught exceptions in middlewareARROW-6281 - [Python] Produce chunked arrays for nested types in pyarrow.arrayARROW-6282 - [Format] Support lossy compressionARROW-6437 - [R] Add AWS SDK to system dependencies for macOS and WindowsARROW-6535 - [C++] Status::WithMessage should accept variadic parametersARROW-6537 - [R] Pass column_types to CSV readerARROW-6972 - [C#] Should support StructField arraysARROW-6982 - [R] Add bindings for compare and boolean kernelsARROW-7136 - [Rust][CI] Pre-install the rust dependencies in the dockerfileARROW-7218 - [Python] Conversion from boolean numpy scalars not workingARROW-7302 - [C++] CSV: allow converting a column to a specific dictionary typeARROW-7372 - [C++] Allow creating dictionary array from simple JSONARROW-7871 - [Python] Expose more compute kernelsARROW-7960 - [C++][Parquet] Add support for schema translation from parquet nodes back to arrow for missing typesARROW-8001 - [R][Dataset] Bindings for dataset writingARROW-8002 - [C++][Dataset] Dataset writing should let you (re)partition the dataARROW-8048 - [Python] Run memory leak tests nightly as follow up to ARROW-4120ARROW-8172 - [C++] ArrayFromJSON for dictionary arraysARROW-8205 - [Rust] [DataFusion] DataFusion should enforce unique field names in a schemaARROW-8253 - [Rust] [DataFusion] Improve ergonomics of registering UDFsARROW-8262 - [Rust] [DataFusion] Add example that uses LogicalPlanBuilderARROW-8289 - [Rust] [Parquet] Implement minimal Arrow Parquet writer as starting point for full writerARROW-8296 - [C++][Dataset] IpcFileFormat should support writing files with compressed buffersARROW-8355 - [Python] Reduce the number of pandas dependent test cases in test_featherARROW-8359 - [C++/Python] Enable aarch64/ppc64le build in conda recipesARROW-8383 - [Rust] Easier random access to DictionaryArray keys and valuesARROW-8402 - [Java] Support ValidateFull methods in JavaARROW-8423 - [Rust] [Parquet] Serialize arrow schema into metadata when writing parquetARROW-8426 - [Rust] [Parquet] Add support for writing dictionary typesARROW-8493 - [C++] Create unified schema resolution code for Array reconstruction.ARROW-8494 - [C++] Implement basic array-by-array reassembly logicARROW-8581 - [C#] Date32/64Array.Builder should accept DateTime, not DateTimeOffsetARROW-8601 - [Go][Flight] Implement Flight Writer interfaceARROW-8601 - [Go][Flight] Implement Flight Writer interfaceARROW-8618 - [C++] ASSIGN_OR_RAISE should move its argumentARROW-8678 - [C++][Parquet] Remove legacy arrow to level translation.ARROW-8712 - [R] Expose strptime timestamp parsing in read_csv conversion optionsARROW-8774 - [Rust] [DataFusion] Improve threading modelARROW-8810 - [R] Add documentation about Parquet format, appending to stream formatARROW-8824 - [Rust] [DataFusion] Implement new SQL parserARROW-8828 - [Rust] Implement SQL tokenizerARROW-8829 - [Rust] Implement SQL parserARROW-9010 - [Java] Framework and interface changes for RecordBatch IPC buffer compressionARROW-9065 - [C++] Support parsing date32 in dataset partition foldersARROW-9068 - [C++][Dataset] Simplify Partitioning interfaceARROW-9078 - [C++] Parquet writing of extension type with nested storage type failsARROW-9104 - [C++] Parquet encryption tests should write files to a temporary directory instead of the testing submodule's directoryARROW-9107 - [C++][Dataset] Time-based types supportARROW-9147 - [C++][Dataset] Support null -> other type promotion in Dataset scanningARROW-9205 - [Documentation] Fix typos in Columnar.rstARROW-9266 - [Python][Packaging] Enable S3 support in macOS wheelsARROW-9271 - [R] Preserve data frame metadata in round tripARROW-9286 - [C++] Add function "aliases" to compute::FunctionRegistryARROW-9328 - [C++][Gandiva] Add LTRIM, RTRIM, BTRIM functions for stringARROW-9338 - [Rust] Add instructions for running clippy locallyARROW-9344 - [C++][Flight] measure latency quantile in flight benchmarkARROW-9358 - [Integration] Reconsider generated_large_batch.jsonARROW-9371 - [Java] Run vector tests for both allocatorsARROW-9377 - [Java] Support unsigned dictionary indicesARROW-9387 - [R] Use new C++ table select methodARROW-9388 - [C++] Division kernelsARROW-9394 - [Python] Support pickling of ScalarsARROW-9398 - [C++] Register the SIMD sum variants under function instance instead a SIMD functionARROW-9402 - [C++] Add portable wrappers for __builtin_add_overflow and friendsARROW-9405 - [R] Switch to cpp11ARROW-9412 - [C++] Add non-BUNDLED dependencies to exported INSTALL_INTERFACE_LIBS of arrow_static and test that it worksARROW-9429 - [Python] ChunkedArray.to_numpyARROW-9454 - [GLib] Add binding of some dictionary buildersARROW-9465 - [Python] Improve ergonomics of compute functionsARROW-9469 - [Python] Make more objects weakrefableARROW-9487 - [Developer] Cover the archery release utilities with unittestsARROW-9488 - [Release] Use the new changelog generation when updating the websiteARROW-9507 - [Rust] [DataFusion] PhysicalExpr should implement Display traitARROW-9508 - [Release][APT][Yum] Enable verification for arm64 binariesARROW-9516 - [Rust][DataFusion] Refactor physical expressions to not care about their names nor indexesARROW-9517 - [C++][Python] Allow session_token argument when initializing S3FileSystemARROW-9518 - [Python] Deprecate pyarrow serializationARROW-9521 - [Rust] CsvReadOptions should allow file extension to be specifiedARROW-9523 - [Rust] improve performance of filter kernelARROW-9534 - [Rust] [DataFusion] Implement functions for creating literal expressions for all typesARROW-9550 - [Rust] [DataFusion] Remove Rc> from hash aggregate operator
ARROW-9553 - [Rust] Release script doesn't bump parquet crate's arrow dependency versionARROW-9557 - [R] Iterating over parquet columns is slow in RARROW-9559 - [Rust] [DataFusion] Revert privatization of exprlist_to_fieldsARROW-9563 - [Dev][Release] Use archery's changelog generator when creating release notes for the websiteARROW-9568 - [CI] Use official msys action on GHAARROW-9576 - [Python][Doc] Fix error in code example for extension typesARROW-9580 - [JS] Docs have superfluous ()ARROW-9581 - [Dev][Release] Bump next snapshot versions to 2.0.0ARROW-9582 - [Rust] Implement Array::memory_size()ARROW-9585 - [Rust] Remove duplicated to-do line in DataFusion readmeARROW-9587 - [FlightRPC][Java] Clean up DoPut/FlightStream memory handlingARROW-9593 - [Python] Add custom pickle reducers for DictionaryScalarARROW-9604 - [C++] Add benchmark for aggregate min/max compute kernelsARROW-9605 - [C++] Optimize performance for aggregate min/max compute kernelsARROW-9607 - [C++][Gandiva] Add bitwise_and(), bitwise_or() and bitwise_not() functions for integersARROW-9608 - [Rust] Remove arrow flight from parquet's feature gatingARROW-9615 - [Rust] Add kernel to compute length of string arrayARROW-9617 - [Rust] [DataFusion] Add length of string arrayARROW-9618 - [Rust] [DataFusion] Make it easier to write optimizersARROW-9619 - [Rust] [DataFusion] Add predicate push-downARROW-9632 - [Rust] Add a "new" method for ExecutionContextSchemaProviderARROW-9638 - [C++][Compute] Implement mode(most frequent number) kernelARROW-9639 - [Ruby] Add dependency version checkARROW-9640 - [C++][Gandiva] Implement round() for integers and long integersARROW-9641 - [C++][Gandiva] Implement round() for floating point and double floating point numbersARROW-9645 - [Python] Deprecate the legacy pyarrow.filesystem interfaceARROW-9646 - [C++][Dataset] Add support for writing parquet datasetsARROW-9650 - [Packaging][APT] Drop support for Ubuntu 19.10ARROW-9654 - [Rust][DataFusion] Add an EXPLAIN command to the datafusion CLIARROW-9656 - [Rust][DataFusion] Slightly confusing error message when unsupported type is provided to CREATE EXTERNAL TABLEARROW-9658 - [Python][Dataset] Bindings for dataset writingARROW-9665 - [R] head/tail/take for DatasetsARROW-9667 - [CI][Crossbow] Segfault in 2 nightly R buildsARROW-9671 - [C++] BasicDecimal128 constructor interprets uint64_t integers with highest bit set as negativeARROW-9673 - [Rust] Add a param "dialect" for DFParser::parse_sqlARROW-9678 - [Rust] [DataFusion] Improve projection push down to remove unused columnsARROW-9679 - [Rust] [DataFusion] HashAggregate walks map many times building final batchARROW-9681 - [Java] Failed Arrow Memory - Core on big-endian platformARROW-9683 - [Rust][DataFusion] Implement Debug for ExecutionPlan traitARROW-9691 - [Rust] [DataFusion] Make sql_statement_to_plan publicARROW-9695 - [Rust][DataFusion] Improve documentation on LogicalPlan variantsARROW-9699 - [C++][Compute] Improve mode kernel performance for small integer typesARROW-9701 - [Java][CI] Add a test job on s390xARROW-9702 - [C++] Move bpacking simd to runtime pathARROW-9703 - [Developer][Archery] Restartable cherry-picking process for creating maintenance branchesARROW-9706 - [Java] Tests in TestLargeListVector fails on big endian platformARROW-9710 - [C++] Generalize Decimal ToString in preparation for Decimal256ARROW-9711 - [Rust] Add benchmark based on TPC-HARROW-9713 - [Rust][DataFusion] Remove explicit panicsARROW-9715 - [R] changelog/doc updates for 1.0.1ARROW-9718 - [Python] Make pyarrow.parquet work with the new filesystem interfacesARROW-9721 - [Packaging][Python] Update wheel dependency filesARROW-9722 - [Rust]: Shorten key lifetime for reverse lookup for dictionary arraysARROW-9723 - [C++] Expected behaviour of "mode" kernel with NaNs ?ARROW-9725 - [Rust] [DataFusion] LimitExec and SortExec should use MergeExecARROW-9737 - [C++][Gandiva] Add bitwise_xor() for integersARROW-9739 - [CI][Ruby] Don't install gem documentsARROW-9742 - [Rust] Create one standard DataFrame APIARROW-9751 - [Rust] [DataFusion] Extend UDFs to accept more than one type per argumentARROW-9752 - [Rust] [DataFusion] Add support for Aggregate UDFsARROW-9753 - [Rust] [DataFusion] Remove the use of Mutex in ExecutionPlan traitARROW-9754 - [Rust] [DataFusion] Implement async in DataFusion traitsARROW-9757 - [Rust] [DataFusion] Use "pub use" to expose a clean public APIARROW-9758 - [Rust] [DataFusion] Implement extension API for DataFusionARROW-9759 - [Rust] [DataFusion] Implement DataFrame::sortARROW-9760 - [Rust] [DataFusion] Implement DataFrame::explainARROW-9761 - [C++] Add experimental pull-based iterator structures to C interface implementationARROW-9762 - [Rust] [DataFusion] ExecutionContext::sql should return DataFrameARROW-9769 - [Python] Remove skip for in-memory fsspec in test_move_fileARROW-9775 - [C++] Automatic S3 region selectionARROW-9781 - [C++] Fix uninitialized value warningsARROW-9782 - [C++][Dataset] Ability to write ".feather" files with IpcFileFormatARROW-9784 - [Rust] [DataFusion] Improve instructions for running tpch benchmarkARROW-9786 - [R] Unvendor cpp11 before releaseARROW-9788 - Handle naming inconsistencies between SQL, DataFrame API and struct namesARROW-9792 - [Rust] [DataFusion] Logical aggregate functions should not return ResultARROW-9794 - [C++] Add functionality to cpu_info to discriminate between Intel vs AMD x86ARROW-9795 - [C++][Gandiva] Implement castTIMESTAMP(int64) in GandivaARROW-9806 - [R] More compute kernel bindingsARROW-9807 - [R] News update/version bump post-1.0.1ARROW-9808 - [Python] parquet.read_table docstring wrong use_legacy_dataset explanationARROW-9811 - [C++] Unchecked floating point division by 0 should succeedARROW-9813 - [C++] Disable semantic interpositionARROW-9819 - [C++] Bump mimalloc to 1.6.4ARROW-9821 - [Rust][DataFusion] User Defined PlanNode / Operator APIARROW-9821 - [Rust][DataFusion] User Defined PlanNode / Operator APIARROW-9823 - [CI][C++][MinGW] Enable S3ARROW-9832 - [Rust] [DataFusion] Refactor PhysicalPlan to remove PartitionARROW-9833 - [Rust] [DataFusion] Refactor TableProvider.scan to return ExecutionPlanARROW-9834 - [Rust] [DataFusion] Remove Partition traitARROW-9835 - [Rust] [DataFusion] Remove FunctionMetaARROW-9836 - [Rust] [DataFusion] Improve API for usage of UDFsARROW-9837 - [Rust] Add provider for variableARROW-9838 - [Rust] [DataFusion] DefaultPhysicalPlanner should insert explicit MergeExec nodesARROW-9839 - [Rust] [DataFusion] Add ability to downcast ExecutionPlan to specific operatorARROW-9841 - [Rust] Update checked-in flatbuffer filesARROW-9844 - [Go][CI] Add Travis CI job for Go on s390xARROW-9845 - [Rust] [Parquet] serde_json is only used in tests but isn't in dev-dependenciesARROW-9848 - [Rust] Implement changes to ensure flatbuffer alignmentARROW-9849 - [Rust] [DataFusion] Make UDFs not need a FieldARROW-9850 - [Go] Defer should not be used in the loopARROW-9853 - [RUST] Implement "take" kernel for dictionary arraysARROW-9854 - [R] Support reading/writing data to/from S3ARROW-9858 - [C++][Python][Docs] Expand user guide for FileSystemARROW-9863 - [C++] [PARQUET] Optimize meta data recovery of ApplicationVersionARROW-9867 - [C++][Dataset] FileSystemDataset should expose its filesystemARROW-9868 - [C++] Provide utility for copying files between filesystemsARROW-9869 - [R] Implement full S3FileSystem/S3Options constructorARROW-9870 - [R] Friendly interface for filesystems (S3)ARROW-9871 - [C++] Add uppercase support to ARROW_USER_SIMD_LEVEL.ARROW-9873 - [C++][Compute] Improve mode kernel for intergers within limited value rangeARROW-9875 - [Python] Let FileSystem.get_file_info accept a single pathARROW-9884 - [R] Bindings for writing datasets to ParquetARROW-9885 - [Rust] [DataFusion] Simplify code of type coercion for binary typesARROW-9886 - [Rust] [DataFusion] Simplify code to test castARROW-9887 - [Rust] [DataFusion] Add support for complex return types of built-in functionsARROW-9890 - [R] Add zstandard compression codec in macOS buildARROW-9891 - [Rust] [DataFusion] Make math functions support f32ARROW-9892 - [Rust] [DataFusion] Add support for concatARROW-9893 - [Python] Bindings for writing datasets to ParquetARROW-9895 - [RUST] Improve sort kernelsARROW-9899 - [Rust] [DataFusion] Switch from Box --> SchemaRef (Arc) to be consistent with the rest of Arrow
ARROW-9900 - [Rust][DataFusion] Use Arc<> instead of Box<> in LogicalPlanARROW-9901 - [C++] Add hand-crafted Parquet to Arrow reconstruction test for nested readingARROW-9902 - [Rust] [DataFusion] Add support for array()ARROW-9904 - [C++] Unroll the loop manually for CountSetBitsARROW-9908 - [Rust] Support temporal data types in JSON readerARROW-9910 - [Rust] [DataFusion] Type coercion of Variadic is wrongARROW-9914 - [Rust][DataFusion] Document the SQL -> Arrow type mappingARROW-9916 - [RUST] Avoid cloning ArrayData in several placesARROW-9917 - [Python][Compute] Add bindings for mode kernelARROW-9919 - [Rust] [DataFusion] Math functionsARROW-9921 - [Rust] Add `from(Vec>)` to [Large]StringArray
ARROW-9925 - [GLib] Add low level value readers for GArrowListArray familyARROW-9926 - [GLib] Use placement new for GArrowRecordBatchFileReaderARROW-9928 - [C++] Speed up integer parsing slightlyARROW-9929 - [Developer] Autotune cmake-formatARROW-9933 - [Developer] Add drone as a CI provider for crossbowARROW-9934 - [Rust] Shape and stride check in tensorARROW-9941 - [Python] Better string representation for extension typesARROW-9944 - [Rust] Implement TO_TIMESTAMP functionARROW-9949 - [C++] Generalize Decimal128::FromString for reuse in Decimal256ARROW-9950 - [Rust] [DataFusion] Allow UDF usage without registryARROW-9952 - [Python] Use pyarrow.dataset writing for pq.write_to_datasetARROW-9954 - [Rust] [DataFusion] Simplify code of aggregate planningARROW-9956 - [C++][Gandiva] Implement Binary string function in GandivaARROW-9957 - [Rust] Remove unmaintained tempdir dependencyARROW-9961 - [Rust][DataFusion] to_timestamp function parses timestamp without timezone offset as UTC rather than localARROW-9964 - [C++] CSV date supportARROW-9965 - [Java] Buffer capacity calculations are slow for fixed-width vectorsARROW-9966 - [Rust] Speedup aggregate kernelsARROW-9967 - [Python] Add compute module docsARROW-9971 - [Rust] Speedup takeARROW-9977 - [Rust] Add min/max for [Large]StringARROW-9979 - [Rust] Fix arrow crate clippy lintsARROW-9980 - [Rust] Fix parquet crate clippy lintsARROW-9981 - [Rust] Allow configuring flight IPC with IpcWriteOptionsARROW-9983 - [C++][Dataset][Python] Use larger default batch size than 32K for Datasets APIARROW-9984 - [Rust] [DataFusion] DRY of function to stringARROW-9986 - [Rust][DataFusion] TO_TIMESTAMP function erroneously requires fractional seconds when no timezone is presentARROW-9987 - [Rust] [DataFusion] Improve docs of `Expr`.ARROW-9988 - [Rust] [DataFusion] Added std::ops to logical expressionsARROW-9992 - [C++][Python] Refactor python to arrow conversions based on a reusable conversion APIARROW-9998 - [Python] Support pickling DictionaryScalarARROW-9999 - [Python] Support constructing dictionary array directly through pa.array()ARROW-10000 - [C++][Python] Support constructing StructArray from list of key-value pairsARROW-10001 - [Rust] [DataFusion] Add developer guide to READMEARROW-10010 - [Rust] Speedup arithmeticARROW-10015 - [Rust] Implement SIMD for aggregate kernel sumARROW-10016 - [Rust] [DataFusion] Implement IsNull and IsNotNullARROW-10018 - [CI] Disable Sphinx and API documentation build since it takes 6 hours on masterARROW-10019 - [Rust] Add substring kernelARROW-10023 - [Gandiva][C++] Implementing Split part function in gandivaARROW-10024 - [C++][Parquet] Create nested reading benchmarksARROW-10028 - [Rust] Simplify macro def_numeric_from_vecARROW-10030 - [Rust] Support fromIter and toIterARROW-10035 - [C++] Bump versions of vendored codeARROW-10037 - [C++] Workaround to force find AWS SDK to look for shared librariesARROW-10040 - [Rust] Create a way to slice unalligned offset buffersARROW-10043 - [Rust] [DataFusion] Introduce support for DISTINCT by partially implementing COUNT(DISTINCT)ARROW-10044 - [Rust] Improve READMEARROW-10046 - [Rust] [DataFusion] Made `*Iterator` implement IteratorARROW-10050 - [C++][Gandiva] Implement concat() in Gandiva for up to 10 argumentsARROW-10051 - [C++][Compute] Make aggregate kernel merge state mutableARROW-10054 - [Python] Slice methods should return empty arrays instead of crashingARROW-10055 - [Rust] Implement DoubleEndedIterator for NullableIterARROW-10057 - [C++] Add Parquet-Arrow roundtrip tests for nested dataARROW-10058 - [C++] Investigate performance of LevelsToBitmap without BMI2ARROW-10059 - [R][Doc] Give more advice on how to set up C++ buildARROW-10063 - [Archery][CI] Fetch main branch in archery build only when it is a pull requestARROW-10064 - [C++] Resolve compile warnings on Apple Clang 12ARROW-10065 - [Rust] DRY downcasted ArraysARROW-10066 - [C++] Make sure that default AWS region is respectedARROW-10068 - [C++] Add bundled external project for aws-sdk-cppARROW-10069 - [Java] Support running Java benchmarks from command lineARROW-10070 - [C++][Compute] Implement stdev aggregate kernelARROW-10071 - [R] segfault with ArrowObject from previous session, or savedARROW-10074 - [C++] Don't use string_view.to_string()ARROW-10075 - [C++] Don't use nonstd::nullopt this breaks out vendoring abstraction.ARROW-10076 - [C++] Use TemporaryDir for all tests that don't already use it.ARROW-10077 - [C++] Potential overflow in bit_stream_utils.h multiplication.ARROW-10083 - [C++] Improve Parquet fuzz seed corpusARROW-10084 - [Rust] [DataFusion] Add length of large string arrayARROW-10086 - [Rust] Migrate min_large_string -> min_string kernelsARROW-10090 - [C++][Compute] Improve mode kernelARROW-10092 - [Dev][Go] Add grpc generated go files to rat exclusion listARROW-10093 - [R] Add ability to opt-out of int64 -> int demotionARROW-10095 - [Rust] [Parquet] Update for IPC changesARROW-10096 - [Rust] [DataFusion] Remove unused codeARROW-10099 - [C++][Dataset] Also allow integer partition fields to be dictionary encodedARROW-10100 - [C++][Dataset] Ability to read/subset a ParquetFileFragment with given set of row group idsARROW-10102 - [C++] Generalize BasicDecimal128::operator*= for reuse in Decimal256ARROW-10103 - [Rust] Add a Contains kernelARROW-10105 - [FlightRPC] Add client option to disable certificate validation with TLSARROW-10120 - [C++][Parquet] Create reading benchmarks for 2-level nested dataARROW-10127 - [Format] Update specification to support 256-bit Decimal typesARROW-10129 - [Rust] Cargo build is rebuilding dependencies on arrow changesARROW-10134 - [C++][Dataset] Add ParquetFileFragment::num_row_groups propertyARROW-10139 - [C++] Add support for building arrow_testing without building testsARROW-10148 - [Rust] Add documentation to lib.rsARROW-10151 - [Python] Add support MapArray to_pandas conversionARROW-10155 - [Rust] [DataFusion] Add documentation to lib.rsARROW-10156 - [Rust] Auto-label PRsARROW-10157 - [Rust] Add more documentation about takeARROW-10160 - [Rust] Improve documentation of DictionaryTypeARROW-10161 - [Rust] [DataFusion] Simplify expression testsARROW-10162 - [Rust] Support display of DictionaryArrays in pretty printingARROW-10164 - [Rust] Add support for DictionaryArray types to cast kernelsARROW-10167 - [Rust] Support display of DictionaryArrays in sql.rsARROW-10168 - [Rust] [Parquet] Extend arrow schema conversion to projected fieldsARROW-10171 - [Rust] [DataFusion] Add `ExecutionContext::from`
ARROW-10190 - [Website] Add Jorge to list of committersARROW-10191 - [Rust] [Parquet] Add roundtrip tests for single column batchesARROW-10196 - [C++] Add Future::DeferNotOk()ARROW-10199 - [Rust][Parquet] Release Parquet at crates.io to remove debug printsARROW-10201 - [C++][CI] Disable S3 in arm64 job on Travis CIARROW-10202 - [CI][Windows] Use sf.net mirror for MSYS2ARROW-10205 - [Java][FlightRPC] Add client option to disable server verificationARROW-10206 - [Python][C++][FlightRPC] Add client option to disable server validationARROW-10215 - [Rust] [DataFusion] Rename "Source" typedefARROW-10217 - [CI] Run fewer GitHub Actions jobsARROW-10225 - [Rust] [Parquet] Fix null bitmap comparisons in roundtrip testsARROW-10227 - [Ruby] Use a table size as the default for parquet chunk_sizeARROW-10229 - [C++][Parquet] Remove left over ARROW_LOG statement.ARROW-10231 - [CI] Unable to download minio in arm32v7 docker imageARROW-10233 - [Rust] Make array_value_to_string available in all Arrow buildsARROW-10235 - [Rust][DataFusion] Improve documentation for type coercionARROW-10240 - [Rust] [Datafusion] Optionally load tpch data into memory before running benchmark queryARROW-10251 - [Rust] [DataFusion] MemTable::load() should load partitions in parallelARROW-10252 - [Python] Add option to skip inclusion of Arrow headers in Python installationARROW-10256 - [C++][Flight] Disable -Werror carefullyARROW-10257 - [R] Prepare news/docs for 2.0 releaseARROW-10260 - [Python] Missing MapType to Pandas dtypeARROW-10265 - [CI] Use smaler build when cache doesn't exit on Travis CIARROW-10266 - [CI][macOS] Ensure using Python 3.8 with HomebrewARROW-10267 - [Python] Skip flight test if disable_server_verification feature is not availableARROW-10272 - [Packaging][Python] Pin newer multibuild version to avoid updating homebrewARROW-10273 - [CI][Homebrew] Fix "brew audit" usageARROW-10287 - [C++] Avoid std::random_device whenever possiblePARQUET-1845 - [C++] Int96 memory images in test cases assume only little-endianPARQUET-1878 - [C++] lz4 codec is not compatible with Hadoop Lz4CodecPARQUET-1904 - [C++] Export file_offset in RowGroupMetaDataApache Arrow 1.0.0 (2020-07-20)
Bug Fixes
ARROW-1692 - [Python, Java] UnionArray round trip not workingARROW-3329 - [Python] Error casting decimal(38, 4) to int64ARROW-3861 - [Python] ParquetDataset().read columns argument always returns partition columnARROW-4018 - [C++] RLE decoder may not big-endian compatibleARROW-4309 - [Documentation] Add a docker-compose entry which builds the documentation with CUDA enabledARROW-4600 - [Ruby] Arrow::DictionaryArray#[] should returns the item in the indices arrayARROW-5158 - [Packaging][Wheel] Symlink libraries in wheelsARROW-5310 - [Python] better error message on creating ParquetDataset from empty directoryARROW-5359 - [Python] timestamp_as_object support for pa.Table.to_pandas in pyarrowARROW-5572 - [Python] raise error message when passing invalid filter in parquet readingARROW-5666 - [Python] Underscores in partition (string) values are dropped when reading datasetARROW-5744 - [C++] Do not error in Table::CombineChunks for BinaryArray types that overflow 2GB limitARROW-5875 - [FlightRPC] Test RPC features in integration testsARROW-6235 - [R] Conversion from arrow::BinaryArray to R character vector not implementedARROW-6523 - [C++][Dataset] arrow_dataset target does not depend on anythingARROW-6848 - [C++] Specify -std=c++11 instead of -std=gnu++11 when buildingARROW-7018 - [R] Non-UTF-8 data in Arrow <--> R conversionARROW-7028 - [R] Date roundtrip results in different R storage modeARROW-7084 - [C++] ArrayRangeEquals should check for full type equality?ARROW-7173 - [Integration] Add test to verify Map field names can be arbitraryARROW-7208 - [Python] Passing directory to ParquetFile class gives confusing error messageARROW-7273 - [Python] Non-nullable null field is allowed / crashes when writing to parquetARROW-7480 - [Rust] [DataFusion] Query fails/incorrect when aggregated + grouped columns don't match the selected columnsARROW-7610 - [Java] Finish support for 64 bit int allocationsARROW-7654 - [Python] Ability to set column_types to a Schema in csv.ConvertOptions is undocumentedARROW-7681 - [Rust] Explicitly seeking a BufReader will discard the internal bufferARROW-7702 - [C++][Dataset] Provide (optional) deterministic order of batchesARROW-7782 - [Python] Losing index information when using write_to_dataset with partition_colsARROW-7840 - [Java] [Integration] Java executables failARROW-7925 - [C++][Documentation] Instructions about running IWYU and other tasks in cpp/development.rst have gone staleARROW-7939 - [Python] crashes when reading parquet file compressed with snappyARROW-7967 - [CI][Crossbow] Pin macOS version in autobrew job to match CRANARROW-8050 - [Python][Packaging] Do not include generated Cython source files in wheel packagesARROW-8078 - [Python] Missing links in the docs regarding field and schema DataTypesARROW-8115 - [Python] Conversion when mixing NaT and datetime objects not workingARROW-8251 - [Python] pandas.ExtensionDtype does not survive round trip with write_to_datasetARROW-8344 - [C#] StringArray.Builder.Clear() corrupts subsequently-built array contentsARROW-8360 - [C++][Gandiva] Fixes date32 support for date/time functionsARROW-8374 - [R] Table to vector of DictonaryType will error when Arrays don't have the same Dictionary per arrayARROW-8392 - [Java] Fix overflow related corner cases for vector value comparisonARROW-8448 - [Package] Can't build apt packages with ubuntu-focalARROW-8455 - [Rust] [Parquet] Arrow column read on partially compatible filesARROW-8455 - [Rust] [Parquet] Arrow column read on partially compatible filesARROW-8471 - [C++][Integration] Regression to /u?int64/ as JSON::numberARROW-8472 - [Go][Integration] Represent 64 bit integers as JSON::stringARROW-8473 - [Rust] "Statistics support" in rust/parquet readme is incorrectARROW-8480 - [Rust] There is no check for allocation failureARROW-8503 - [Packaging][deb] Can't build apache-arrow-archive-keyring for RCARROW-8505 - [Release][C#] "sourcelink test" is failed by Apache.Arrow.AssemblyInfo.csARROW-8508 - [Rust] ListBuilder of FixedSizeListBuilder creates wrong offsetsARROW-8510 - [C++] arrow/dataset/file_base.cc fails to compile with internal compiler error with "Visual Studio 15 2017 Win64" generatorARROW-8511 - [Developer][Release] Windows release verification script does not halt if C++ compilation failsARROW-8514 - [Developer] Windows wheel verification script does not check Python 3.5ARROW-8529 - [C++] Fix usage of NextCounts() in GetBatchWithDict[Spaced]ARROW-8535 - [Rust] Arrow crate does not specify arrow-flight versionARROW-8536 - [Rust] Failed to locate format/Flight.proto in any parent directoryARROW-8537 - [C++] Performance regression from ARROW-8523ARROW-8539 - [CI] "AMD64 MacOS 10.15 GLib & Ruby" failsARROW-8554 - [C++][Benchmark] Fix building error "cannot bind lvalue"ARROW-8556 - [R] zstd symbol not found if there are multiple installations of zstdARROW-8566 - [R] error when writing POSIXct to sparkARROW-8568 - [C++][Python] Crash on decimal cast in debug modeARROW-8577 - [Plasma] PlasmaClient::Connect() of CUDA enabled build is always failed on no CUDA device machineARROW-8583 - [C++][Doc] Undocumented parameter in Dataset namespaceARROW-8584 - [Packaging][C++] Protobuf link error in deb buildsARROW-8585 - [Packaging][Python] Windows wheels fail to build because of link errorARROW-8586 - [R] installation failure on CentOS 7ARROW-8587 - [C++] Compilation error when linking arrow-flight-perf-serverARROW-8592 - [C++] Docs still list LLVM 7 as compiler usedARROW-8593 - [C++] Parquet file_serialize_test.cc fails to build with musl libcARROW-8598 - [Rust] simd_compare_op creates buffer of incorrect length when item count is not a multiple of T::lanes()ARROW-8602 - [CMake] Fix ws2_32 link issue when cross-compiling on LinuxARROW-8603 - [Documentation] Fix Sphinx doxygen commentARROW-8604 - [R][CI] Update CI to use R 4.0ARROW-8608 - [C++] Update vendored mpark/variant.h to latest to fix NVCC compilation issuesARROW-8609 - [C++] ORC JNI bridge crashed on null arrow bufferARROW-8610 - [Rust] DivideByZero when running arrow crate when simd feature is disabledARROW-8613 - [C++][Dataset] Raise error for unparsable partition valueARROW-8615 - [R] Error better and insist on RandomAccessFile in read_featherARROW-8617 - [Rust] simd_load_set_invalid does not exist on aarch64ARROW-8632 - [C++] Fix conversion error warning in array_union_test.ccARROW-8641 - [Python] Regression in feather: no longer supports permutation in column selectionARROW-8643 - [Python] Tests with pandas master failing due to freq assertionARROW-8644 - [Python] Dask integration tests failing due to change in not including partition columnsARROW-8646 - [Java] Allow UnionListWriter to write null valuesARROW-8649 - [Java] [Website] Java documentation on website is hiddenARROW-8657 - [Python][C++][Parquet] Forward compatibility issue from 0.16 to 0.17 when using version='2.0'ARROW-8663 - [Documentation] Small correction to building.rstARROW-8680 - [Rust] ComplexObjectArrayReader incorrect null value shufflingARROW-8684 - [Python] "SystemError: Bad call flags in _PyMethodDef_RawFastCallDict" in Python 3.7.7 on macOS when using pyarrow wheelARROW-8689 - [C++] S3 benchmarks fail linkingARROW-8693 - [Python] Dataset.get_fragments is missing an implicit cast when filteringARROW-8694 - [Python][Parquet] parquet.read_schema() fails when loading wide table created from Pandas DataFrameARROW-8701 - [Rust] Unresolved import `crate::compute::util::simd_load_set_invalid` on Raspberry PiARROW-8704 - [C++] Fix Parquet crash on invalid input (OSS-Fuzz)ARROW-8705 - [Java] ComplexCopier is skipping null valuesARROW-8706 - [C++][Parquet] Tracking JIRA for PARQUET-1857 (unencrypted INT16_MAX Parquet row group limit)ARROW-8710 - [Rust] Continuation marker not written correctly in IPC writer, and stream not flushedARROW-8722 - [Dev] "archery docker run -e" doesn't workARROW-8726 - [C++][Dataset] Mis-specified DirectoryPartitioning incorrectly uses the file name as valueARROW-8728 - [C++] Bitmap operation may cause buffer overflowARROW-8729 - [C++][Dataset] Only selecting a partition column results in empty tableARROW-8734 - [R] improve nightly build installationARROW-8741 - [Python][Packaging] Keep VS2015 with for the windows wheelsARROW-8750 - [Python] pyarrow.feather.write_feather does not default to lz4 compression if it's availableARROW-8768 - [R][CI] Fix nightly as-cran spurious failureARROW-8775 - [C++][FlightRPC] Integration client doesn't run integration testsARROW-8776 - [FlightRPC][C++] Flight/C++ middleware don't receive headers on failed calls to Java serversARROW-8798 - [C++] Fix Parquet crashes on invalid input (OSS-Fuzz)ARROW-8799 - [C++][Dataset] Reading list column as nested dictionary segfaultsARROW-8801 - [Python] Memory leak on read from parquet file with UTC timestamps using pandasARROW-8802 - [C++][Dataset] Schema metadata are lost when reading a subset of columnsARROW-8803 - [Java] Row count should be set before loading buffers in VectorLoaderARROW-8808 - [Rust] Divide by zero in arrays/builder.rsARROW-8809 - [Rust] schema mismatch in integration testARROW-8811 - [Java] Fix build on masterARROW-8820 - [C++][Gandiva] fix date_trunc functions to return date typesARROW-8821 - [Rust] nested binary expression with Like, NotLike and Not operator results in type cast errorARROW-8825 - [C++] Cannot compiled pass with Wunused-parameter flagARROW-8826 - [Crossbow] remote URL should always have .gitARROW-8832 - [Python] AttributeError: module 'pyarrow.fs' has no attribute 'S3FileSystem'ARROW-8848 - [CI][C/Glib] MinGW build errorARROW-8848 - [CI][C/Glib] MinGW build errorARROW-8858 - [FlightRPC] Ensure headers are uniformly exposedARROW-8860 - [C++] IPC/Feather decompression broken for nested arraysARROW-8862 - [C++] NumericBuilder does not use MemoryPool passed to CTORARROW-8863 - [C++] Array subclass constructors must set ArrayData::null_count to 0 when there is no validity bitmapARROW-8869 - [Rust] [DataFusion] Type Coercion optimizer rule does not support new scan nodesARROW-8871 - [C++] Gandiva build failureARROW-8872 - [CI] Travis-CI jobs fail (can't open file 'ci/detect-changes.py')ARROW-8874 - [C++][Dataset] Scanner::ToTable race when ScanTask exit early with an errorARROW-8878 - [R] try_download is confused when download.file.method isn't defaultARROW-8882 - [C#] Add .editorconfig to C# codeARROW-8888 - [Python] Heuristic in dataframe_to_arrays that decides to multithread convert cause slow conversionsARROW-8889 - [Python] Python 3.7 SIGSEGV when comparing RecordBatch to NoneARROW-8892 - [C++][CI] CI builds for MSVC do not build benchmarksARROW-8909 - [Java] Out of order writes using setSafeARROW-8911 - [C++] Slicing a ChunkedArray with zero chunks segfaultsARROW-8924 - [C++][Gandiva] castDATE_date32() may cause overflowARROW-8925 - [Rust] [DataFusion] CsvExec::schema() returns incorrect resultsARROW-8930 - [C++] libz.so linking error with liborc.aARROW-8932 - [C++] symbol resolution failures with liborc.aARROW-8946 - [Python] Add tests for parquet.write_metadata metadata_collectorARROW-8948 - [Java][Integration] enable duplicate field names integration testsARROW-8951 - [C++] Fix compiler warning in compute/kernels/scalar_cast_temporal.ccARROW-8954 - [Website] ca-certificates should be listed in installation instructionsARROW-8957 - [FlightRPC][C++] Fail to build due to IpcOptionsARROW-8959 - [Rust] Broken build due to new benchmark crate using old APIARROW-8962 - [C++] Linking failure with clang-4.0ARROW-8968 - [C++][Gandiva] Show link warning message on s390xARROW-8975 - [FlightRPC][C++] Fix flaky MacOS testsARROW-8977 - [R] Table$create with schema crashes with some dictionary index typesARROW-8978 - [C++][Compute] "Conditional jump or move depends on uninitialised value(s)" Valgrind warningARROW-8980 - [Python] Metadata grows exponentially when using schema from diskARROW-8982 - [CI] Remove allow_failures for s390x in TravisCIARROW-8986 - [Archery][ursabot] Fix benchmark diff checkout of origin/masterARROW-9000 - [Java] build crashes with JDK14ARROW-9009 - [C++][Dataset] ARROW:schema should be removed from schema's metadata when reading Parquet filesARROW-9013 - [C++] Validate enum-style CMake optionsARROW-9020 - [Python] read_json won't respect explicit_schema in parse_optionsARROW-9024 - [C++/Python] Install anaconda-client in conda-clean jobARROW-9026 - [C++/Python] Force package removal from arrow-nightlies conda repositoryARROW-9037 - [C++][C] unable to import array with null count == -1 (which could be exported)ARROW-9057 - [Rust] Projection should work on InMemoryScan without errorARROW-9059 - [Rust] Documentation for slicing array data has the wrong signARROW-9066 - [Python] Raise correct error in isnull()ARROW-9071 - [C++] MakeArrayOfNull makes invalid ListArrayARROW-9077 - [C++] Fix aggregate/scalar-compare benchmark null_percent calculationARROW-9080 - [C++] arrow::AllocateBuffer returns a Result>
ARROW-9082 - [Rust] - Stream reader fail when steam not ended with (optional) 0xFFFFFFFF 0x00000000"ARROW-9084 - [C++] CMake is unable to find zstd target when ZSTD_SOURCE=SYSTEMARROW-9085 - [C++][CI] Appveyor CI test failuresARROW-9087 - [C++] Missing HDFS options parsingARROW-9098 - RecordBatch::ToStructArray cannot handle record batches with 0 columnARROW-9105 - [C++] ParquetFileFragment scanning doesn't handle filter on partition fieldARROW-9120 - [C++] Lint and Format C++ files with "codegen" in file nameARROW-9121 - [C++] Do not wipe the filesystem when path is emptyARROW-9122 - [C++] Adapt ascii_lower/ascii_upper bulk transforms to work on sliced arraysARROW-9126 - [C++] Trimmed Boost bundle fails to build on WindowsARROW-9127 - [Rust] Update thrift library dependenciesARROW-9134 - [Python] Parquet partitioning degrades Int32 to float64ARROW-9141 - [R] Update cross-package documentation linksARROW-9142 - [C++] random::RandomArrayGenerator::Boolean "probability" misdocumented / incorrectARROW-9143 - [C++] RecordBatch::Slice erroneously sets non-nullable field's internal null_count to unknownARROW-9146 - [C++][Dataset] Scanning a Fragment with a filter + mismatching schema shouldn't abortARROW-9151 - [R][CI] Fix Rtools 4.0 build: pacman syncARROW-9160 - [C++] Implement string/binary contains for exact matchesARROW-9174 - [Go] Tests panic with 386 (x86) buildsARROW-9183 - [C++] Failed to build arrow-cpp with gcc 4.9.2ARROW-9184 - [Rust][Datafusion] table scan without projection should return all columnsARROW-9194 - [C++] Array::GetScalar not implemented for decimal typeARROW-9195 - [Java] Wrong usage of Unsafe.get from bytearray in ByteFunctionsHelper classARROW-9209 - [C++] Benchmarks fail to build ARROW_IPC=OFF and ARROW_BUILD_TESTS=OFFARROW-9219 - [R] coerce_timestamps in Parquet write options does not workARROW-9221 - ArrowBuf#setBytes(int, ByteBuffer) doesn't check the byte buffer's endiannessARROW-9223 - [Python] Fix to_pandas() export for timestamps within structsARROW-9230 - [FlightRPC][Python] flight.connect() doesn't pass through all argumentsARROW-9233 - [C++] is_null on NullArray should be true for all valuesARROW-9236 - [Rust] CSV WriterBuilder never writes headerARROW-9237 - [R] 0.17 install on Arch LinuxARROW-9238 - [C++][CI] A few test coverages of round-robin in ipc and flightARROW-9252 - [Integration] GitHub Actions integration test job does not test against "gold" 0.14.1 files in apache/arrow-testingARROW-9260 - [CI] "ARM64v8 Ubuntu 20.04 C++" failsARROW-9260 - [CI] "ARM64v8 Ubuntu 20.04 C++" failsARROW-9261 - [Python][Packaging] S3FileSystem curl errors in manylinux wheelsARROW-9274 - [Rust] [Integration Testing] Read i64 from json files as stringsARROW-9282 - [R] Remove usage of _EXTPTR_PTRARROW-9284 - [Java] getMinorTypeForArrowType returns sparse minor type for dense union typesARROW-9288 - [C++][Dataset] Discovery of partition field as dictionary type segfaulting with HivePartitioningARROW-9297 - [C++][Dataset] Dataset scanner cannot handle large binary column (> 2 GB)ARROW-9298 - [C++] Fix crashes on invalid input (OSS-Fuzz)ARROW-9303 - [R] Linux static build should always bundle dependenciesARROW-9305 - [Python] Dependency load failure in Windows wheel buildARROW-9315 - [Java] Fix the failure of testAllocationManagerTypeARROW-9317 - [Java] A few testcases for arrow-memoryARROW-9326 - [Python] Setuptools 49.1.0 appears to break our Python 3.6 buildsARROW-9326 - [Python] Setuptools 49.1.0 appears to break our Python 3.6 buildsARROW-9326 - [Python] Setuptools 49.1.0 appears to break our Python 3.6 buildsARROW-9330 - [C++] Fix crashes on corrupt IPC input (OSS-Fuzz)ARROW-9334 - [Dev][Archery] Push ancestor docker imagesARROW-9336 - [Ruby] Creating RecordBatch with structs missing keys results in a malformed tableARROW-9343 - [C++][Gandiva] CastINT/Float functions from string should handle leading/trailing white spacesARROW-9347 - [Python] Tests fail with latest fsspecARROW-9350 - [C++][CI] Nightly valgrind job failuresARROW-9351 - [C++][CI] Nightly test-ubuntu-18.04-cpp-cmake32 failsARROW-9353 - [Python][CI] Nightly dask integration jobs failARROW-9354 - [C++] Turbodbc latest fails to build in the integration testsARROW-9355 - [R] Fix -Wimplicit-int-float-conversionARROW-9360 - [CI][Crossbow] Nightly homebrew-cpp job times outARROW-9363 - [C++][Dataset] ParquetDatasetFactory schema: pandas metadata is lostARROW-9368 - [Python] Rename predicate argument to filter in split_by_row_group()ARROW-9373 - [C++] Fix Parquet crash on invalid input (OSS-Fuzz)ARROW-9380 - [C++] Segfaults in compute::CallFunctionARROW-9384 - [C++] Out-of-memory on invalid IPC input (OSS-Fuzz)ARROW-9385 - [Python] [CI] jpype integration failureARROW-9389 - [C++] Can't call isin/match through CallFunctionARROW-9397 - [R] Pass CC/CXX to cmake when building libarrow in Linux buildARROW-9408 - [Integration] Tests do not run in Windows due to numpy 64-bit errorsARROW-9409 - [CI][Crossbow] Nightly conda-r failsARROW-9410 - [CI][Crossbow] Fix homebrew-cpp againARROW-9413 - [Rust] Fix clippy lint on masterARROW-9415 - [C++] Arrow does not compile on Power9ARROW-9416 - [Go] Add test cases for some datatypesARROW-9417 - [C++][IPC] size in message written in native endianARROW-9418 - [R] nyc-taxi Parquet files not downloaded in binary mode on WindowsARROW-9419 - [C++] Test that "fill_null" function works with sliced inputs, expand testsARROW-9428 - [C++] Update documentation for buffer allocation functionsARROW-9436 - [C++][CI] Valgrind errors in fill_null kernel testsARROW-9438 - [CI] Spark integration tests are failingARROW-9439 - [C++] Fix crash on invalid IPC input (OSS-Fuzz)ARROW-9440 - [Python] Expose Fill Null Compute Kernel in PyArrowARROW-9443 - [C++] Bundled bz2 build should only build libbz2ARROW-9448 - [Java] Circular initialization between ArrowBuf and BaseAllocator leads to null HistoricalLog for empty bufferARROW-9449 - [R] Strip arrow.soARROW-9450 - [Python] "pytest pyarrow" takes over 10 seconds to collect tests and start executingARROW-9456 - [Python] Dataset segfault when not importing pyarrow.parquetARROW-9458 - [Python] Dataset Scanner is single-threaded onlyARROW-9460 - [C++] BinaryContainsExact doesn't cope with double characters in the patternARROW-9461 - [Rust] Reading Date32 and Date64 errors - they are incorrectly converted to RecordBatchARROW-9476 - [C++][Dataset] HivePartitioning discovery with dictionary types fails for multiple fieldsARROW-9486 - [C++][Dataset] Support implicit casting InExpression::set_ to dictARROW-9497 - [C++][Parquet] Fix failure caused by malformed repetition/definition levelsARROW-9499 - [C++] AdaptiveIntBuilder::AppendNull does not increment the null countARROW-9500 - [C++] Fix segfault with std::to_string in -O3 builds on gcc 7.5.0ARROW-9501 - [C++][Gandiva] Add logic in timestampdiff() when end date is last day of a monthARROW-9503 - [Rust] Comparison sliced arrays is wrongARROW-9504 - [Python] Segmentation fault on ChunkedArray.takeARROW-9506 - [Packaging][Python] Fix macOS wheel build failuresARROW-9512 - [C++] Variadic template unpack inside lambda doesn't compile with gccARROW-9524 - [CI][Gandiva] C++ unit test arrow-ipc-read-write failing in gandiva nightly buildARROW-9527 - [Rust] Remove un-needed dev-dependenciesARROW-9528 - [Python] Honor tzinfo information when converting from datetime to pyarrowPARQUET-1839 - [C++] values_read not updated in ReadBatchSpacedPARQUET-1857 - [C++][Parquet] ParquetFileReader unable to read files with more than 32767 row groupsPARQUET-1865 - [C++] Failure from C++17 feature used in parquet/encoding_benchmark.ccPARQUET-1877 - [C++] Reconcile container size with string size for memory issuesPARQUET-1882 - [C++] Writing an all-null column and then reading it with buffered_stream aborts the processNew Features and Improvements
ARROW-300 - [Format] Add body buffer compression option to IPC message protocol using LZ4 or ZSTDARROW-842 - [Python] Handle more kinds of null sentinel objects from pandas 0.xARROW-971 - [C++/Python] Implement Array.isvalid/notnull/isnull as scalar functionsARROW-974 - [Website] Add Use Cases section to the websiteARROW-1277 - Completing integration tests for major implemented data typesARROW-1567 - [C++] Implement "fill null" kernels that replace null values with some scalar replacement valueARROW-1570 - [C++] Define API for creating a kernel instance from function of scalar input and output with a particular signatureARROW-1682 - [Python] Add documentation / example for reading a directory of Parquet files on S3ARROW-1796 - [Python] RowGroup filtering on file levelARROW-2260 - [C++][Plasma] plasma_store should show usageARROW-2444 - [Python][C++] Better handle reading empty parquet filesARROW-2702 - [Python] Examine usages of Invalid and TypeError errors in numpy_to_arrow.cc to see if we are using the right error type in each instanceARROW-2714 - [C++/Python] Variable step size slicing for arraysARROW-2912 - [Website] Build more detailed Community landing page a la Apache SparkARROW-3089 - [Rust] Add ArrayBuilder for different Arrow arraysARROW-3134 - [C++] Implement n-ary iterator for a collection of chunked arrays with possibly different chunking layoutsARROW-3154 - [Python][C++] Document how to write _metadata, _common_metadata files with Parquet datasetsARROW-3244 - [Python] Multi-file parquet loading without scanARROW-3275 - [Python] Add documentation about inspecting Parquet file metadataARROW-3308 - [R] Convert R character vector with data exceeding 2GB to Large typeARROW-3317 - [R] Test/support conversions from data.frame with a single character column exceeding 2GB capacity of BinaryArrayARROW-3446 - [R] Document mapping of Arrow <-> R typesARROW-3509 - [C++] Inconsistent child accessor namingARROW-3520 - [C++] Implement List Flatten kernelARROW-3688 - [Rust] Implement PrimitiveArrayBuilder.push_values
ARROW-3827 - [Rust] Implement UnionArrayARROW-4022 - [C++] Promote Datum variant out of compute namespaceARROW-4221 - [Format] Add canonical flag in COO sparse indexARROW-4390 - [R] Serialize "labeled" metadata in Feather files, IPC messagesARROW-4412 - [DOCUMENTATION] Add explicit version numbers to the arrow specification documents.ARROW-4427 - [Doc] Move Confluence Wiki pages to the Sphinx docsARROW-4429 - [Doc] Add git rebase tips to the 'Contributing' page in the developer docsARROW-5035 - [C#] ArrowBuffer.Builder is broken
ARROW-5082 - [Python][Packaging] Reduce size of macOS and manylinux1 wheelsARROW-5143 - [Flight] Enable integration testing of batches with dictionariesARROW-5279 - [C++] Support reading delta dictionaries in IPC streamsARROW-5377 - [C++] Make IpcPayload public and add GetPayloadSizeARROW-5489 - [C++] Normalize kernels and ChunkedArray behaviorARROW-5548 - [Documentation] http://arrow.apache.org/docs/latest/ is not latestARROW-5649 - [Integration][C++] Create round trip integration test for extension typesARROW-5708 - [C#] Null support for BooleanArrayARROW-5760 - [C++] Optimize Take implementationARROW-5854 - [Python] Expose compare kernels on Array classARROW-6052 - [C++] Divide up arrow/array.h,cc into files in arrow/array/ similar to builder filesARROW-6110 - [Java] Support LargeList Type and add integration test with C++ARROW-6111 - [Java] Support LargeVarChar and LargeBinary types and add integration test with C++ARROW-6439 - [R] Implement S3 file-system interface in RARROW-6456 - [C++] Possible to reduce object code generated in compute/kernels/take.cc?ARROW-6501 - [C++] Remove non_zero_length field from SparseIndexARROW-6521 - [C++] Add function to arrow:: namespace that returns the current ABI versionARROW-6543 - [R] Support LargeBinary and LargeString typesARROW-6602 - [Doc] Add feature / implementation matrixARROW-6603 - [C#] ArrayBuilder API to support writing nullsARROW-6645 - [Python] Faster boundschecking of dictionary indices when converting to CategoricalARROW-6689 - [Rust] [DataFusion] Query execution enhancements for 1.0.0 releaseARROW-6691 - [Rust] [DataFusion] Use tokio and Futures instead of spawning threadsARROW-6775 - [C++] [Python] Proposal for several Array utility functionsARROW-6776 - [Python] Need a lite version of pyarrowARROW-6800 - [C++] Add CMake option to build libraries targeting a C++14 or C++17 toolchain environmentARROW-6839 - [Java] Add APIs to read and write "custom_metadata" field of IPC file footerARROW-6856 - [C++] Use ArrayData instead of Array for ArrayData::dictionaryARROW-6917 - ARROW-6917: [Archery][Release] Add support for JIRA curation, changelog generation and commit cherry-picking for maintenance releasesARROW-6945 - [Rust] Enable integration testsARROW-6959 - [C++] Clarify what signatures are preferred for compute kernelsARROW-6978 - [R] Add bindings for sum and mean compute kernelsARROW-6979 - [R] Enable jemalloc in autobrew formulaARROW-7009 - [C++] Refactor filter/take kernels to use Datum instead of overloadsARROW-7010 - [C++] Support lossy casts from decimal128 to float32 and float64/doubleARROW-7011 - [C++] Implement casts from float/double to decimal128ARROW-7012 - [C++] Clarify ChunkedArray chunking strategy and policyARROW-7068 - [C++] Expose the offsets of a ListArray as a Int32ArrayARROW-7075 - [C++] Boolean kernels should not allocate in Call()ARROW-7175 - [Website] Add a security page to track when vulnerabilities are patchedARROW-7229 - [C++] Unify ConcatenateTables APIsARROW-7230 - [C++] Use vendored std::optional instead of boost::optional in GandivaARROW-7237 - [C++] Add Result to APIs to arrow/json
ARROW-7243 - [Docs] Add common "implementation status" table to the README of each native language implementation, as well as top level READMEARROW-7285 - [C++] ensure C++ implementation meets clarified dictionary specARROW-7300 - [C++][Gandiva] Implement functions to cast from strings to integers/floatsARROW-7313 - [C++] Add function for retrieving a scalar from an array slotARROW-7371 - [GLib] Add Datasets bindingARROW-7375 - [Python] Expose C++ MakeArrayOfNullARROW-7391 - [Python] Remove unnecessary classes from the binding layerARROW-7495 - [Java] Remove "empty" concept from ArrowBuf, replace with custom referencemanagerARROW-7605 - [C++] Create and install static library containing all dependencies built by ArrowARROW-7607 - [C++] Add to cpp/examples minimal examples of using Arrow as a dependency of another CMake projectARROW-7673 - [C++][Dataset] Revisit File discovery failure modeARROW-7676 - [Packaging][Python] Ensure that the static libraries are not built in the wheel scriptsARROW-7699 - [Java] Support concating dense union vectors in batchARROW-7705 - [Rust] Initial sort implementationARROW-7717 - [CI] Have nightly integration test for Spark's latest releaseARROW-7759 - [C++][Dataset] Add CsvFileFormat for CSV supportARROW-7778 - [C++] Support nested dictionaries in JSON integration formatARROW-7784 - [C++] diff.cc is extremely slow to compileARROW-7801 - [Developer] Add issue_comment workflow to fix lint/style/codegenARROW-7803 - [R][CI] Autobrew/homebrew tests should not always install from masterARROW-7831 - [Java] unnecessary buffer allocation when calling splitAndTransferTo on variable width vectorsARROW-7831 - [Java] unnecessary buffer allocation when calling splitAndTransferTo on variable width vectorsARROW-7902 - [Integration] Unskip nested dictionary integration testsARROW-7910 - [C++] Provide function to query page size portablyARROW-7924 - [Rust] Add sort for float typesARROW-7950 - [Python] When initializing pandas API shim, inform user if their installed pandas version is too oldARROW-7955 - [Java] Support large buffer for file/stream IPCARROW-8020 - [Java] Implement vector validate functionalityARROW-8023 - [Website] Write a blog post about the C data interfaceARROW-8025 - [C++] Implement cast to Binary and FixedSizeBinaryARROW-8025 - [C++] Implement cast to Binary and FixedSizeBinaryARROW-8046 - [Developer][Integration] Makefile.docker's target names are brokenARROW-8062 - [C++][Dataset] Parquet Dataset factory from a _metadata/_common_metadata fileARROW-8065 - [C++][Dataset] Untangle Dataset, Fragment and ScanOptionsARROW-8074 - [C++][Dataset] Support for file-like objects (buffers) in FileSystemDataset?ARROW-8108 - [Java] Extract a common interface for dictionary encodersARROW-8111 - [C++][CSV] Support MM/DD/YYYY date formatARROW-8114 - [Java][Integration] Enable custom_metadata integration testARROW-8121 - [Java] Enhance code style checking for Java code (add space after commas, semi-colons and type casts)ARROW-8149 - [C++/Python] Enable CUDA Support in conda recipesARROW-8157 - [C++][Gandiva] Support building with LLVM 9ARROW-8162 - [Format][Python] Add serialization for CSF sparse tensorsARROW-8169 - [Java] Improve the performance of JDBC adapter by allocating memory proactivelyARROW-8171 - Consider pre-allocating memory for fix-width vector in Avro adapter iteratorARROW-8190 - [C++][Flight] Allow setting IpcWriteOptions and IpcReadOptions in Flight IPC message reader and writer classesARROW-8229 - [Java] Move ArrowBuf into the Arrow packageARROW-8230 - [Java] Move Netty memory manager into a separate moduleARROW-8261 - [Rust] [DataFusion] LogicalPlanBuilder.limit() should take a literal argumentARROW-8263 - [Rust] [DataFusion] Add documentation for supported SQL functionsARROW-8281 - [R] Name collision of arrow.dll on Windows condaARROW-8283 - [Python][Dataset] Non-existent files are silently dropped in pa.dataset.FileSystemDatasetARROW-8287 - [Rust] Arrow examples should use utility to print resultsARROW-8293 - [Python] Run flake8 on python/examples alsoARROW-8297 - [FlightRPC][C++] Implement Flight DoExchange for C++ARROW-8301 - [R] Handle ChunkedArray and Table in C data interfaceARROW-8312 - [Java][Gandiva] improve IN expression supportARROW-8314 - [Python] Provide a method to select a subset of columns of a TableARROW-8318 - [C++][Dataset] Dataset should instantiate FragmentARROW-8399 - [Rust] Extend memory alignments to include other architecturesARROW-8413 - [C++] Refactor DefLevelsToBitmapARROW-8422 - [Rust] [Parquet] Implement function to convert Arrow schema to Parquet schemaARROW-8430 - [CI] Configure self-hosted runners for Github ActionsARROW-8434 - [C++] Ipc RecordBatchFileReader deserializes the Schema multiple timesARROW-8440 - [C++] Refine simd header filesARROW-8443 - [Gandiva][C++] Fix round/truncate to no-op for special casesARROW-8447 - [C++][Dataset] Ensure Scanner::ToTable preserve ordering of ScanTasksARROW-8467 - [C++] Test cases using ArrayFromJSON assume only a little-endian platformARROW-8474 - [CI][Crossbow] Skip some nightlies we don't need to runARROW-8477 - [C++] Enable reading and writing of long filenames for WindowsARROW-8481 - [Java] Provide an allocation manager based on Unsafe APIARROW-8483 - [Ruby] Arrow::Table documentation improvementARROW-8485 - [Integration][Java] Implement extension types integrationARROW-8486 - [C++] arrow-utility-test causes failures on a big-endian platformARROW-8487 - [FlightRPC][C++] Make it possible to target a specific payload sizeARROW-8488 - [R] Replace VALUE_OR_STOP with ValueOrStopARROW-8496 - [C++] Refine ByteStreamSplitDecodeScalarARROW-8497 - [Archery] Add missing component to buildsARROW-8499 - [C++][Dataset] In ScannerBuilder, batch_size will not work if projecter is not emptyARROW-8500 - [C++] Use selection vectors in Filter implementation for record batches, tablesARROW-8501 - [Packaging][RPM] Upgrade devtoolset to 8 on CentOS 6ARROW-8502 - [Release][APT][Yum] Ignore all arm64 verificationsARROW-8504 - [C++] Add Run Length ReaderARROW-8506 - [c++] Miss tests to verify expected_buffer with bit_width > 8 in RLEARROW-8507 - [Release] Detect .git directory automatically in changelog.pyARROW-8509 - [GLib] Add low level record batch read/write functionsARROW-8512 - [C++] Delete unused compute expr prototype codeARROW-8513 - [Python] Expose Take with Table input in PythonARROW-8515 - [C++] Bitmap ToString should have an option of grouping by bytesARROW-8516 - [Rust] Slow BufferBuilder inserts within PrimitiveBuilder::append_slice
ARROW-8517 - [Developer][Release] Update Crossbow RC verification setup for changes since 0.16.0ARROW-8520 - [Developer] Use .asf.yaml to direct GitHub notifications to e-mail lists and JIRAARROW-8521 - [Developer] Group Sub-task, Task, Test, and Wish issue types as "Improvement" in ChangelogARROW-8522 - [Developer] Add environment variable option to toggle whether ephemeral NodeJS is installed in release verification scriptARROW-8524 - [CI] Free up space on github actionsARROW-8526 - [Python] Fix non-deterministic row order failure in dataset testsARROW-8531 - [C++] Deprecate ARROW_USE_SIMD CMake optionARROW-8538 - [Packaging] Remove boost from homebrew formulaARROW-8540 - [C++] Create memory allocation benchmarkARROW-8541 - [Release] Don't remove previous source releases automaticallyARROW-8542 - [Release] Fix checksum url in the website post release scriptARROW-8543 - [C++] IO: single pass coalescing algorithmARROW-8544 - [CI][Crossbow] Add a status.json to the gh-pages summary of nightly builds to get around rate limitingARROW-8548 - [Website] 0.17 release postARROW-8549 - [R] Assorted post-0.17 release cleanupsARROW-8550 - [CI] Don't run cron GHA jobs on forksARROW-8551 - [CI][Gandiva] Use LLVM 8 to build gandiva linux jarARROW-8552 - [Rust] support column iteration for parquet rowARROW-8553 - [C++] Optimize unaligned bitmap operationsARROW-8555 - [FlightRPC][Java] Implement Flight DoExchange for JavaARROW-8558 - [Rust] GitHub Actions missing rustfmtARROW-8559 - [Rust] Consolidate Record Batch reader traits in main arrow crateARROW-8560 - [Rust] Docs for MutableBuffer resize are incorrectARROW-8561 - [C++][Gandiva] Stop using deprecated google::protobuf::MessageLite::ByteSize()ARROW-8562 - [C++] IO: Parameterize I/O coalescing using S3 storage metricsARROW-8563 - [Go] Minor change to make newBuilder publicARROW-8564 - [Website] Add Ubuntu 20.04 LTS to supported package listARROW-8569 - [CI] Upgrade xcode version for testing homebrew formulaeARROW-8571 - [C++] Switch AppVeyor image to VS 2017ARROW-8572 - [Python] Expose UnionArray.array and other fieldsARROW-8573 - [Rust] Upgrade to Rust 1.44 nightlyARROW-8574 - [Rust] Implement Debug for all plain typesARROW-8575 - [Developer] Add issue_comment workflow to rebase a PRARROW-8590 - [Rust] Use Arrow pretty print utility in DataFusionARROW-8591 - [Rust] Reverse lookup for a key in DictionaryArrayARROW-8597 - [Rust] arrow crate lint and readability improvementsARROW-8606 - [CI] Don't trigger all builds on a change to any file in ci/ARROW-8607 - [R][CI] Unbreak builds following R 4.0 releaseARROW-8611 - [R] Can't install arrow 0.17 on Ubuntu 18.04 R 3.6.3ARROW-8612 - [GLib] Add GArrowReadOptions and GArrowWriteOptionsARROW-8616 - [Rust] Turn explicit SIMD off by defaultARROW-8619 - [C++] Use distinct Type::type values for interval typesARROW-8622 - [Rust] Parquet crate does not compile on aarch64ARROW-8623 - [C++][Gandiva] Reduce use of Boost, remove Boost headers from header filesARROW-8624 - [Website] Install page should mention arrow-dataset packagesARROW-8628 - [CI][Dev] Wrap docker-compose commands with archeryARROW-8629 - [Rust] Eliminate indirection of ZST allocationsARROW-8633 - [C++] Add ValidateAscii functionARROW-8634 - [Java] Create an exampleARROW-8639 - [C++][Plasma] Require gflagsARROW-8645 - [C++] Missing gflags dependency for plasmaARROW-8647 - [C++][Dataset] Optionally encode partition field values as dictionary typeARROW-8648 - [Rust] Optimize Rust CI Build TimesARROW-8650 - [Rust] [Website] Add documentation to Arrow websiteARROW-8651 - [Python][Dataset] Support pickling of Dataset objectsARROW-8655 - [C++][Dataset][Python][R] Preserve partitioning information for a discovered DatasetARROW-8656 - [Python] Switch to VS2017 in the windows wheel buildsARROW-8659 - [Rust] ListBuilder and FixedSizeListBuilder capacityARROW-8660 - [C++][Gandiva] Reduce dependence on BoostARROW-8662 - [CI] Consolidate appveyor scriptsARROW-8664 - [Java] Add skip null check to all Vector typesARROW-8668 - [Packaging][APT][Yum][ARM] Use Travis CI's ARM machine to build packagesARROW-8669 - [C++] Add IpcWriteOptions argument to GetRecordBatchSize()ARROW-8671 - [C++] Use IPC body compression metadata approved in ARROW-300ARROW-8671 - [C++] Use IPC body compression metadata approved in ARROW-300ARROW-8682 - [Ruby][Parquet] Add support for column level compressionARROW-8687 - [Java] Finish move of io.netty.buffer.ArrowBufARROW-8690 - [Python] Clean-up dataset+parquet tests now order is determinsticARROW-8692 - [C++] Avoid memory copies when downloading from S3ARROW-8695 - [Java] remove references to PlatformDependent in memory moduleARROW-8696 - [Java] Convert tests to integration testsARROW-8699 - [R] Fix automatic r_to_py conversionARROW-8702 - [Packaging][C#] Build NuGet packages in release processARROW-8703 - [R] schema$metadata should be properly typedARROW-8707 - [CI] Docker push fails because of wrong dockerhub credentialsARROW-8708 - [CI] Utilize github actions cache for docker-compose volumesARROW-8711 - [Python] Expose strptime timestamp parsing in read_csv conversion optionsARROW-8717 - [CI][Packaging] Add build dependency on boost to homebrewARROW-8720 - [C++] Fix checked_pointer_castARROW-8721 - [CI] Fix R build matrixARROW-8723 - [Rust] Remove SIMD specific benchmark codeARROW-8724 - [Packaging][deb][RPM] Use directory in host as build directoryARROW-8725 - [Rust] redundant directory walk in rust parquet datasource codeARROW-8727 - [C++] Do not require struct-initialization of StringConverter to parse strings to other types
ARROW-8730 - [Rust] Use slice instead of &Vec for function argumentsARROW-8733 - [C++][Dataset][Python] ParquetFileFragment should provide access to parquet FileMetadataARROW-8736 - [Rust] [DataFusion] Table API should provide a schema() methodARROW-8740 - [CI] Fix archery option in pandas master cron testARROW-8742 - [C++][Python] Add flight client support for Mutual TLSARROW-8743 - [C++][CI] Add a test job on s390xARROW-8744 - [Rust] ParquetIterator's next method should be safe to call even after reached end of iterationARROW-8745 - [C++] Bitmap.ToString causes failures on a big-endian platformARROW-8747 - [C++] Feather tests with compression cause failure on big-endian platformsARROW-8751 - [Rust] ParquetFileArrowReader should be able to read empty parquet file without errorARROW-8752 - [Rust] Remove unused hashmapARROW-8753 - [C++][CI] Add a test job on ARMARROW-8754 - [C++][CI] enable tests for additional components on big-endian platformsARROW-8756 - [C++] Bitmap word tests cause failures on a big-endian platformARROW-8757 - [C++] Plasma header is written in native endianARROW-8758 - [R] Updates for compatibility with dplyr 1.0ARROW-8759 - [C++] TestPlasmaSerialization.DeleteReply tests failure on big-endian platformsARROW-8762 - [C++][Gandiva] Replace Gandiva's BitmapAnd with common implementationARROW-8763 - [C++] Create RandomAccessFile::WillNeed-like APIARROW-8764 - [C++] Make ThreadPool configurable in ReadRangeCacheARROW-8766 - [Python] A FileSystem implementation based on Python callbacksARROW-8769 - [C++] Add convenience methods to access fields by name in StructScalarARROW-8770 - [C++][CI] enable arrow-csv-test on s390xARROW-8772 - [C++] Expand SumKernel benchmark to more typesARROW-8777 - [Rust] Parquet.rs does not support reading fixed-size binary fields.ARROW-8778 - [C++][Gandiva] SelectionVector related test failed on big-endian platformsARROW-8779 - [R] Implement conversion to List
ARROW-8781 - [CI][C++] Enable ccache on GHA MinGW jobsARROW-8782 - [Rust] [DataFusion] Add benchmarks based on NYC Taxi data setARROW-8783 - [Rust] [DataFusion] Logical plan should have ParquetScan and CsvScan entriesARROW-8784 - [Rust] [DataFusion] Remove use of Arc from LogicalPlanARROW-8785 - [Python][Packaging] Build the windows wheels with MIMALLOC enabledARROW-8786 - [Packaging][rpm] Use bundled zstd in the CentOS 8 buildARROW-8788 - [C#] Array builders to use bit-packed buffer builder rather than boolean array builder for validity mapARROW-8789 - [Rust] Add separate crate for integration test binariesARROW-8790 - [C++][CI] Enable arrow-flight-test on s390xARROW-8791 - [Rust] Creating StringDictionaryBuilder with existing dictionary valuesARROW-8792 - [C++] Improved declarative compute function / kernel development framework, normalize calling conventionsARROW-8793 - [C++] BitUtil::SetBitsTo probably doesn't need to be inlineARROW-8794 - [C++] Expand benchmark coverage for arrow from parquet readingARROW-8795 - [C++] Limited iOS supportARROW-8800 - [C++] Split arrow::ChunkedArray into arrow/chunked_array.hARROW-8804 - [R][CI] Followup to Rtools40 upgradeARROW-8814 - [Dev][Release] Binary upload script keeps raising locale warningsARROW-8815 - [Dev][Release] Binary upload script should retry on unexpected bintray request errorARROW-8818 - [Rust] Failing to build on master due to Flatbuffers/Union issuesARROW-8822 - [Rust] [DataFusion] Add MemoryScan variant to LogicalPlanARROW-8827 - [Integration Testing] Initial skeleton for Rust integration testsARROW-8830 - [GLib] Add support for Tell againt not seekable GIO output streamARROW-8831 - [Rust] incomplete SIMD implementation in simd_compare_opARROW-8833 - [Rust] Implement VALIDATE mode in integration test binaryARROW-8834 - [Rust] Implement arrow-file-to-stream for integration testingARROW-8835 - [Rust] Implement arrow-stream-to-file for integration testingARROW-8836 - [Website] Update copyright end year automaticallyARROW-8837 - [Rust] Add Null typeARROW-8838 - [Rust] File reader fails to read header from valid filesARROW-8839 - [Rust] datafusion logical plan should support scaning csv without provided schemaARROW-8840 - [Rust] datafusion ExecutionError should implement std::error:Error traitARROW-8841 - [C++] Add benchmark and unittest for PLAIN spacedARROW-8843 - [C++] Optimize BitmapEquals unaligned caseARROW-8844 - [C++] Optimize TransferBitmap unaligned caseARROW-8846 - [Dev][Python] Autoformat Python sources with ArcheryARROW-8847 - [C++] Pass task size / metrics in Executor APIARROW-8851 - [Python][Documentation] Fix FutureWarnings in Python Plasma docsARROW-8852 - [R] Post-0.17.1 adjustmentsARROW-8854 - [Rust] [Integration Testing] Show output from arrow-json-integration-testARROW-8855 - [Rust] [Integration Testing] data type Date32(Day) not supportedARROW-8856 - [Rust] [Integration Testing] Return empty batch if MessageHeader is NONEARROW-8864 - [R] Add methods to Table/RecordBatch for consistency with data.frameARROW-8866 - [C++] Split Type::UNION into Type::SPARSE_UNION and Type::DENSE_UNIONARROW-8867 - [R] Support converting POSIXlt typeARROW-8875 - [C++] use AWS SDK SetResponseStreamFactory to avoid a copy of bytesARROW-8877 - [Rust] add CSV read option struct to simplify datafusion interfaceARROW-8880 - [R][Linux] Make R Binary Install FriendlierARROW-8881 - [Rust] Add large list and binary supportARROW-8885 - [R] Don't include everything everywhereARROW-8886 - [C#] Decide and implement appropriate behaviour for Array builder resize to negative sizeARROW-8887 - [Java] Buffer size for complex vectors increases rapidly in case of clear/write loopARROW-8890 - [R] Fix C++ lint issueARROW-8895 - [C++] Add C++ unit tests for filter and take functions on temporal type inputs, including timestampsARROW-8896 - [C++] Reimplement dictionary unpacking in Cast kernels using TakeARROW-8899 - [R] Add R metadata like pandas metadata for round-trip fidelityARROW-8901 - [C++] Reduce number of take kernelsARROW-8903 - [C++] Implement optimized "unsafe take" for use with selection vectors for kernel executionARROW-8904 - [Python] Fix usages of deprecated C++ APIs related to child/fieldARROW-8906 - [Rust] Support reading multiple CSV files for schema inferenceARROW-8907 - [Rust] implement scalar comparison operationsARROW-8912 - [Ruby] Keep reference of Arrow::Buffer's data for GCARROW-8913 - [Ruby] Use "field" instead of "child"ARROW-8914 - [C++][Gandiva] Decimal128 related test failed on big-endian platformsARROW-8915 - [Dev][Archery] Require Click 7ARROW-8917 - [C++][Compute] Formalize "metafunction" conceptARROW-8918 - [C++] Add cast "metafunction" to FunctionRegistry that addresses dispatching to appropriate type-specific CastFunctionARROW-8922 - [C++] Implement example string scalar kernel function to assist with string kernels buildout per ARROW-555ARROW-8923 - [C++] Improve usability of arrow::compute::CallFunction by moving ExecContext* argument to end and adding defaultARROW-8926 - [C++] Improve docstrings in new public APIs in arrow/compute and fix miscellaneous typosARROW-8927 - [C++] Support dictionary memos when reading/writing record batches using cuda IPCARROW-8929 - [C++] Change compute::Arity:VarArgs min_args default to 0ARROW-8931 - [Rust] Support lexical sort in arrow compute kernelARROW-8933 - [C++] Reduce generated code in vector_hash.ccARROW-8934 - [C++] Add timestamp subtract kernel aliased to int64 subtract implementationARROW-8937 - [C++] Add "parse_strptime" function for string to timestamp conversions using the kernels frameworkARROW-8938 - [R] Provide binding for arrow::compute::CallFunctionARROW-8940 - [Java] Fix the performance degradation of integration testsARROW-8941 - [C++/Python] arrow-nightlies conda repository is fullARROW-8942 - [R] Detect compression in reading CSV/JSONARROW-8943 - [C++][Dataset] Add support for Partitioning to ParquetDatasetFactoryARROW-8950 - [C++] Make head optional in s3fsARROW-8958 - [FlightRPC][Python] Implement Flight DoExchange for PythonARROW-8960 - [MINOR] [FORMAT] Fix typos in commentsARROW-8961 - [C++] Add utf8proc library to toolchainARROW-8963 - [C++][Parquet] Parquet cpp optimize allocate memoryARROW-8965 - [Python][Documentation] Pyarrow documentation for pip nightlies references 404'd locationARROW-8966 - [C++] Move arrow::ArrayData to a separate header fileARROW-8969 - [C++] Reduce generated code in compute/kernels/scalar_compare.ccARROW-8970 - [C++] Reduce shared library / binary code size (umbrella issue)ARROW-8972 - [Java] Support range value comparison for large varchar/varbinary vectorsARROW-8973 - [Java] Support batch value appending for large varchar/varbinary vectorsARROW-8974 - [C++] Refine TransferBitmap template parametersARROW-8976 - [C++] compute::CallFunction can't Filter/Take with ChunkedArrayARROW-8979 - [C++] Implement bitmap word reader and writerARROW-8984 - [R] Revise install guides now that Windows conda package existsARROW-8985 - [Format] Add "byte width" field with default of 16 to Decimal Flatbuffers type for forward compatibilityARROW-8989 - [C++] Document available functions in compute::FunctionRegistryARROW-8993 - [Rust] Support reading non-seekable sources in text readersARROW-8994 - [C++] Disable include-what-you-use cpplint lint checksARROW-8996 - [C++] Runtime SIMD path for Aggregate Sum/Mean kernelARROW-8997 - [Archery] Benchmark formatter should have friendly unitsARROW-9004 - [C++][Gandiva] Support building with LLVM 10ARROW-9005 - [Rust] [DataFusion] Support sort expressionARROW-9007 - [Rust] Support appending arrays by merging array dataARROW-9014 - [Packaging] Bump the minor part of the automatically generated version in crossbowARROW-9015 - [Java] Make BaseAllocator package privateARROW-9016 - [Java] Remove direct references to Netty/Unsafe AllocatorsARROW-9017 - [Python] Refactor the Scalar classesARROW-9018 - [C++] Remove APIs that were deprecated in 0.17.x and priorARROW-9021 - [Python] The filesystem keyword in parquet.read_table is not documentedARROW-9022 - [C++] Add/Sub/Mul arithmetic kernels with overflow checkARROW-9029 - [C++] Implement BitBlockCounter interface for blockwise popcounts of validity bitmapsARROW-9030 - [Python] Clean up some usages of pyarrow.compat, move some common functions/symbols to lib.pyxARROW-9031 - [R] Implement conversion from Type::UINT64 to R vectorARROW-9032 - [C++] Split arrow/util/bit_util.h into multiple header filesARROW-9034 - [C++] Implement binary (two bitmap) version of BitBlockCounterARROW-9042 - [C++] Add Subtract and Multiply arithmetic kernels with wrap-around behaviorARROW-9043 - [Go] Temporarily copy LICENSE.txt to go/ARROW-9043 - [Go] Temporarily copy LICENSE.txt to go/ARROW-9045 - [C++] Improve and expand Take/Filter benchmarksARROW-9046 - [C++][R] Put more things in type_fwdsARROW-9047 - [Rust] Setting 0-bits of a 0-length bitset segfaultsARROW-9050 - [Release] Use 1.0.0 as the next versionARROW-9051 - [GLib] Refer Array related objects from ArrayARROW-9052 - [CI][MinGW] Enable GandivaARROW-9055 - [C++] Add sum/mean kernels for Boolean typeARROW-9058 - [Packaging][wheel] Boost download is failedARROW-9060 - [GLib] Add support for building Apache Arrow Datasets GLib with non-installed Apache Arrow DatasetsARROW-9061 - [Packaging][APT][Yum][GLib] Add Apache Arrow Datasets GLibARROW-9062 - [Rust] Support to read JSON into dictionary typeARROW-9067 - [C++] Create reusable branchless / vectorized index boundschecking functionsARROW-9070 - [C++] StructScalar needs field accessor methodsARROW-9073 - [C++] RapidJSON include directory detection doesn't work with RapidJSONConfig.cmakeARROW-9074 - [GLib] Add missing arrow-json checkARROW-9075 - [C++] Optimize Filter implementationARROW-9079 - [C++] Write benchmark for arithmetic kernelsARROW-9083 - [R] collect int64, uint32, uint64 as R integer type if not out of boundsARROW-9086 - [CI][Homebrew] Enable GandivaARROW-9088 - [Rust] Recent version of arrow crate does not compile into wasm targetARROW-9089 - [Python] A PyFileSystem handler for fsspec-based filesystemsARROW-9090 - [C++] Bump versions of bundled librariesARROW-9091 - [C++] Utilize function's default options when passing no options to CallFunction for a function that requires themARROW-9093 - [FlightRPC][C++][Python] Allow setting gRPC client optionsARROW-9094 - [Python] Bump versions of compiled dependencies in manylinux wheelsARROW-9095 - [Rust] Fix NullArray to comply with specARROW-9099 - [C++][Gandiva] Add TRIM function for stringARROW-9100 - [C++] Add ascii_lower kernelARROW-9101 - [Doc][C++][Python] Document encoding expected by CSV and JSON readersARROW-9102 - [Packaging] Upload built manylinux docker imagesARROW-9106 - [C++] Add C++ foundation to ease file transcodingARROW-9108 - [C++][Dataset] Add Parquet Statistics conversion for timestamp columnsARROW-9109 - [Python][Packaging] Enable S3 support in manylinux wheelsARROW-9110 - [C++] Fix CPU cache size detection on macOSARROW-9112 - [R] Update autobrew script locationARROW-9115 - [C++] Process data buffers in batch in ascii_lower / ascii_upper kernels rather than using string_view value iterationARROW-9116 - [C++] Add BinaryArray::total_values_length()ARROW-9116 - [C++] Add BinaryArray::total_values_length()ARROW-9118 - [C++] Add more general BoundsCheck function that also checks for arbitrary lower limits in integer arraysARROW-9119 - [C++] Add support for building with system static gRPCARROW-9123 - [Python][wheel] Use libzstd.a explicitlyARROW-9124 - [Rust][Datafusion] DFParser should consume sql query as &str instead of StringARROW-9125 - [C++] Add missing include for arrow::internal::ZeroMemory() for ValgrindARROW-9129 - [Python][JPype] Test is failed with JPype 0.7.5ARROW-9130 - [Python] Add deprecated wrappers functions to a pyarrow/compat.py module for 1.0.0 that will be removed laterARROW-9131 - [C++] Faster ascii_lower and ascii_upperARROW-9132 - [C++] Implement hash kernels for dictionary data with constant dictionariesARROW-9133 - [C++] Add utf8_upper and utf8_lowerARROW-9137 - [GLib][Ruby] Allow to read Parquet files in chunks (by RowGroup)ARROW-9138 - [Docs][Format] Make sure format version is hard coded in the docsARROW-9139 - [Python] parquet read_table should not use_legacy_datasetARROW-9144 - [CI] OSS-Fuzz build fails because recent changes in the google repositoryARROW-9145 - [C++] Add true_count / false_count methods to BooleanArrayARROW-9152 - [C++] Create specialized filter implementation for varbinary typesARROW-9153 - [Python] Add bindings for StructScalarARROW-9154 - [Developer] Use GitHub issue templates betterARROW-9155 - [Archery] Less precise but faster default settings for "archery benchmark diff"ARROW-9156 - [C++] Reducing the code size of the tensor moduleARROW-9157 - [Rust][Datafusion] execution context's create_physical_plan should take self as immutable referenceARROW-9158 - [Rust][Datafusion] Projection physical plan compilation should preserve nullabilityARROW-9159 - [Python] Expose the isnull/isvalid kernelsARROW-9162 - [Python] Expose Add/Subtract/Multiply arithmetic kernelsARROW-9163 - [C++] Add methods to StringArray, LargeStringArray, to validate whether its values are all UTF-8ARROW-9166 - [Website] Add overview pageARROW-9167 - [Doc][Website] /docs/c_glib/index.html is overwrittenARROW-9168 - [C++][Flight] allow flight benchmark to use separated TCP connectionsARROW-9173 - [C++] Document how to use Arrow from a third-party CMake projectARROW-9175 - [FlightRPC][C++][Python] Expose connected peerARROW-9176 - [Rust] Fix for memory leaks in Arrow allocatorARROW-9178 - [R] Improve documentation about CSV readerARROW-9179 - [R] Replace usage of iris dataset in testsARROW-9180 - [Developer] Remove usage of whitelist, blacklist, slave, etc.ARROW-9181 - [C++] Instantiate fewer templates in Cast kernel implementationARROW-9182 - [C++] Use "applicator" namespace for kernel operator-to-kernel functors, streamline argument unboxingARROW-9185 - [C++] [Java][Gandiva] Make llvm build optimisation configurable from javaARROW-9188 - [C++] Do not always statically link Brotli librariesARROW-9189 - [Website] Improve contributor guideARROW-9190 - [Website][C++] Add blog post on efforts to make building lighter and easierARROW-9191 - [Rust] Do not panic when int96 milliseconds are negativeARROW-9192 - [CI][Rust] Add support for running clippyARROW-9193 - [C++] Add method to parse date from null-terminated stringARROW-9197 - [C++] Revamp numeric casts: faster performance and reduced binary sizeARROW-9201 - [Archery] Render-human readable table when using "archery benchmark diff"ARROW-9202 - [GLib] Add GArrowDatumARROW-9203 - [Packaging][deb] Add missing gir1.2-arrow-dataset-1.0.installARROW-9204 - [C++][Flight] change records_per_stream to int64 in flight benchmarkARROW-9205 - [Documentation] Fix typos in Columnar.rstARROW-9206 - [C++][Flight] measure latency in flight benchmarkARROW-9207 - [Python][Dataset] Clean-up internal FileSource classARROW-9210 - [C++] Use OptionalBitBlockCounter in ArrayDataInlineVisitorARROW-9214 - [C++] Avoid util::optional in favor of separate inlineable functions in arrow/visitor_inline.hARROW-9216 - [C++][Parquet] Use BitBlockCounter for plain spaced encoding/decodingARROW-9217 - [C++][Parquet] Cover 0.01% null for the plain spaced encoding/decoding benchmarkARROW-9220 - [C++] Disable relevant compute kernels if ARROW_WITH_UTF8PROC=OFFARROW-9222 - [Format][Proposal] Remove validity bitmap from Union typesARROW-9224 - [Dev][Archery] Copy local repo on clone failureARROW-9225 - [C++][Compute] Improve counting sortARROW-9231 - [Format] Increment MetadataVersion from V4 to V5ARROW-9234 - [GLib][CUDA] Add support for dictionary memo on reading record batch from bufferARROW-9241 - [C++] Add forward compatibility checks for Decimal::bitWidthARROW-9242 - [Java] Add forward compatibility checks for Decimal::bitWidthARROW-9247 - [Python] Expose BinaryArray::total_values_length in bindingsARROW-9248 - [C++] Add "list_size" function that returns Int32Array/Int64Array giving list cell sizesARROW-9249 - [C++] Implement "list_parent_indices" vector functionARROW-9250 - [C++] Compact generated code in compute/kernels/scalar_set_lookup.cc using same method as vector_hash.ccARROW-9251 - [C++] Move JSON testing code for integration tests to libarrow_testingARROW-9254 - [C++] Factor out some integer casting internals so it can be reused with temporal castsARROW-9255 - [C++] Use CMake to build bundled Protobuf with CMake >= 3.7ARROW-9256 - [C++] Incorrect variable name ARROW_CXX_FLAGSARROW-9258 - [Format] Add V5 MetadataVersionARROW-9259 - [Format] Permit unsigned dictionary indices in Columnar.rstARROW-9262 - [Packaging][Linux][CI] Use Ubuntu 18.04 to build ARM64 packages on Travis CIARROW-9263 - [C++] Benchmark: promote RegressionSetArgs size to L2ARROW-9264 - [C++] Cleanup Parquet Arrow Schema codeARROW-9265 - [C++] Add support for writing MetadataVersion::V4-compatible IPC messages for compatibility with library versions <= 0.17.1ARROW-9268 - [C++] Add is{alnum,alpha,...} kernels for stringsARROW-9272 - [C++][Python] Reduce complexity in python to arrow conversionARROW-9276 - [Dev] Enable ARROW_CUDA when generating API documentationsARROW-9277 - [C++] Fix documentation of Reading CSV filesARROW-9278 - [C++] Implement Union validity bitmap changes from ARROW-9222ARROW-9280 - [Rust] Write statistics to Parquet filesARROW-9281 - [R] Turn off utf8proc in R buildsARROW-9283 - [Python] Expose C++ build infoARROW-9287 - [C++] Implement support for unsigned dictionary indicesARROW-9289 - [R] Remove deprecated functionsARROW-9290 - [Rust] [Parquet] Add features to allow opting out of dependenciesARROW-9291 - [R] Support fixed size binary/list typesARROW-9292 - [Rust] Update feature matrix with passing testsARROW-9294 - [GLib] Add GArrowFunctionARROW-9300 - [Java] Separate Netty Memory to its own moduleARROW-9306 - [Ruby] Add support for Arrow::RecordBatch.new(raw_table)ARROW-9307 - [Ruby] Add Arrow::RecordBatchIterator#to_aARROW-9308 - [Format] Add Feature enum to schema.fbs for forward compatibityARROW-9316 - [C++] Use "Dataset" instead of "Datasets"ARROW-9321 - [C++][Dataset] Allow to "collect" statistics for ParquetFragment row groups if not constructed from _metadataARROW-9322 - [R] Dataset documentation polishingARROW-9323 - [Ruby] Add Red Arrow DatasetARROW-9327 - Fix all clippy errors for arrow crateARROW-9329 - [C++][Gandiva] Implement castTimestampToDate functionARROW-9331 - [C++] Improve the performance of Tensor-to-SparseTensor conversionARROW-9333 - [Python] Expose more IPC write options in PythonARROW-9335 - [Website] Update website for 1.0ARROW-9337 - [R] On C++ library build failure, give an unambiguous messageARROW-9339 - [Rust] Comments on SIMD in Arrow README are incorrectARROW-9340 - [R] Use CRAN version of decor packageARROW-9341 - [GLib] Use arrow::Datum version Take()ARROW-9345 - [C++][Dataset] Expression with dictionary type should work with operand of value typeARROW-9346 - [C++][Python][Dataset] Add total_byte_size metadata to RowGroupInfoARROW-9362 - [Java] Add support for writing MetadataVersion::V4-compatible IPC messages for compatibility with library versions <= 0.17.1ARROW-9365 - [Go] Implement the rest of the typed array builders in NewBuilderARROW-9370 - [Java] Bump Netty versionARROW-9374 - [C++][Python] Expose MakeArrayFromScalarARROW-9379 - [Rust] Support unsigned dictionary indicesARROW-9383 - [Python] Support fsspec filesystems in Dataset API through fs handlerARROW-9386 - [Rust] RecordBatch.schema() should not return &Arc
ARROW-9390 - [C++] Review compute function namesARROW-9390 - [C++] Review compute function namesARROW-9390 - [C++] Review compute function namesARROW-9391 - [Rust] Float32 values interpreted as zero when record batch has one rowARROW-9393 - [Doc] update supported types documentation for JavaARROW-9395 - [Python] Provide configurable MetadataVersion in IPC API and environment variable to set default to V4 when neededARROW-9399 - [C++] Add forward compatibility checks for unrecognized future MetadataVersionARROW-9403 - [Python] add .tolist as alias of .to_pylistARROW-9407 - [Python] Accept pd.NA as missing value in array constructorARROW-9411 - [Rust] Update dependenciesARROW-9424 - [C++][Parquet] Disable writing files with LZ4 codecARROW-9425 - [Rust][DataFusion] Make ExecutionContext sharable between threadsARROW-9427 - [Rust][DataFusion] Add pub fn ExecutionContext.tables()ARROW-9437 - [Python][Packaging] Homebrew fails to install build dependencies in the macOS wheel buildsARROW-9442 - [Python] Do not force Validate() to be called in pyarrow_wrap_tableARROW-9445 - [Python] Revert Array.equals changes + expose comparison ops in computeARROW-9446 - [C++] Export compiler information in BuildInfoARROW-9447 - [Rust][DataFusion] Allow closures as ScalarUDFsARROW-9452 - [Rust] [DateFusion] Improve performance of parquet scanARROW-9470 - [CI][Java] Run Maven in parallelARROW-9472 - [R] Provide configurable MetadataVersion in IPC API and environment variable to set default to V4 when neededARROW-9473 - [Doc] Polishing for 1.0ARROW-9478 - [C++] Improve error message on unsupported cast typesARROW-9484 - [Docs] Update is* functions to be is_* in the compute docsARROW-9485 - [R] Better shared library strippingARROW-9493 - [Python][Dataset] Dictionary encode string partition columns by defaultARROW-9509 - [Release] Don't test Gandiva in the windows wheel verification scriptARROW-9511 - [Packaging][Release] Set conda packages' build number to 0ARROW-9519 - [Rust] Improve error message when getting a field by name from schemaARROW-9523 - [Rust] improve performance of filter kernelARROW-9529 - [Dev][Release] Improvements to release verification scriptsARROW-9531 - [Packaging][Release] Update conda forge dependency pinsPARQUET-1820 - [C++] Use a column filter hint to inform read prefetching in Arrow readsPARQUET-1843 - [C++] Unnecessary assignment in DictDecoderImpl::DecodePARQUET-1855 - [C++] Improve documentation on MetaData ownershipPARQUET-1861 - [Documentation][C++] Explain ReaderProperters.buffer_stream*Apache Arrow 0.17.1 (2020-05-18)
Bug Fixes
ARROW-8503 - [Packaging][deb] Can't build apache-arrow-archive-keyring for RCARROW-8505 - [Release][C#] "sourcelink test" is failed by Apache.Arrow.AssemblyInfo.csARROW-8584 - [Packaging][C++] Protobuf link error in deb buildsARROW-8608 - [C++] Update vendored mpark/variant.h to latest to fix NVCC compilation issuesARROW-8609 - [C++] ORC JNI bridge crashed on null arrow bufferARROW-8641 - [Python] Regression in feather: no longer supports permutation in column selectionARROW-8657 - [Python][C++][Parquet] Forward compatibility issue from 0.16 to 0.17 when using version='2.0'ARROW-8684 - [Python] "SystemError: Bad call flags in _PyMethodDef_RawFastCallDict" in Python 3.7.7 on macOS when using pyarrow wheelARROW-8694 - [Python][Parquet] parquet.read_schema() fails when loading wide table created from Pandas DataFrameARROW-8704 - [C++] Fix Parquet crash on invalid input (OSS-Fuzz)ARROW-8706 - [C++][Parquet] Tracking JIRA for PARQUET-1857 (unencrypted INT16_MAX Parquet row group limit)ARROW-8728 - [C++] Bitmap operation may cause buffer overflowARROW-8741 - [Python][Packaging] Keep VS2015 with for the windows wheelsARROW-8750 - [Python] pyarrow.feather.write_feather does not default to lz4 compression if it's availablePARQUET-1857 - [C++][Parquet] ParquetFileReader unable to read files with more than 32767 row groups
New Features and Improvements
ARROW-8501 - [Packaging][RPM] Upgrade devtoolset to 8 on CentOS 6ARROW-8549 - [R] Assorted post-0.17 release cleanupsARROW-8699 - [R] Fix automatic r_to_py conversionARROW-8758 - [R] Updates for compatibility with dplyr 1.0ARROW-8786 - [Packaging][rpm] Use bundled zstd in the CentOS 8 build
Apache Arrow 0.17.0 (2020-04-20)
Bug Fixes
ARROW-1907 - [C++/Python] Feather format cannot accommodate string columns containing more than a total of 2GB of dataARROW-2255 - [Developer][Integration] Serialize schema- and field-level custom metadata in integration test JSON formatARROW-2587 - [Python] Unable to write StructArrays with multiple children to parquetARROW-3004 - [Documentation] Builds docs for master rather than a pinned commitARROW-3543 - [R] Better support for timestamp format and time zones in RARROW-5265 - [Python/CI] Add integration test with kartothekARROW-5473 - [C++] Build failure on googletest_ep on Windows when using NinjaARROW-5981 - [C++] DictionaryBuilder initialization with Array can fail silently
ARROW-6528 - [C++] Spurious Flight test failures (port allocation failure)ARROW-6547 - [C++] valgrind errors in diff-testARROW-6738 - [Java] Fix problems with current union comparison logicARROW-6757 - [Python] Creating csv.ParseOptions() causes "Windows fatal exception: access violation" with Visual Studio 2017ARROW-6871 - [Java] Enhance TransferPair related parameters check and testsARROW-6872 - [C++][Python] Empty table with dictionary-columns raises ArrowNotImplementedErrorARROW-6890 - [Rust] [Parquet] ArrowReader fails with seg faultARROW-6895 - [C++][Parquet] parquet::arrow::ColumnReader: ByteArrayDictionaryRecordReader repeats returned values when calling `NextBatch()`ARROW-7008 - [Python] pyarrow.chunked_array([array]) fails on array with all-None buffersARROW-7049 - [C++] warnings building on mingw-w64ARROW-7301 - [Java] Sql type DATE should correspond to DateDayVectorARROW-7335 - [C++][Gandiva] Add castBIGINT, extractDay interval_day functions in GandivaARROW-7390 - [C++][Dataset] Concurrency race in Projector::ProjectARROW-7405 - [Java] ListVector isEmpty API is incorrectARROW-7466 - [CI][Java] Fix gandiva-jar-osx nightly build failureARROW-7467 - [Java] ComplexCopier does incorrect copy for Map nullable infoARROW-7507 - [Rust] Bump Thrift version to 0.13 in parquet-format and parquetARROW-7520 - [R] Writing many batches causes a crashARROW-7546 - [Java] Use new implementation to concat vectors values in batchARROW-7624 - [Rust] Soundness issues via `Buffer` methodsARROW-7628 - [Python] Better document some read_csv corner casesARROW-7631 - [C++][Gandiva] return zero if there is an overflow while converting a decimal to a lower precision/scaleARROW-7672 - [C++] NULL pointer dereference bugARROW-7680 - [C++][Dataset] Partition discovery is not working with windows pathARROW-7701 - [C++] [CI] Flight test error on macOSARROW-7713 - [Java] TastLeak was put at the wrong locationARROW-7722 - [Java][FlightRPC] Memory leakARROW-7734 - [C++] Segfault when comparing status with and without detailARROW-7740 - [C++] Array internals corruption in StructArray::FlattenARROW-7755 - [Python] Windows wheel cannot be installed on Python 3.8ARROW-7758 - [Python] Wrong conversion of timestamps that are out of bounds for pandas (eg 0000-01-01)ARROW-7760 - [Release] Fix verify-release-candidate.sh since pip3 seems to no longer be in minicondaARROW-7762 - [Python] Exceptions in ParquetWriter get ignoredARROW-7766 - [Python][Packaging] Windows py38 wheels are built with wrong ABI tagARROW-7772 - [R][C++][Dataset] Unable to filter on date32 object with date64 scalarARROW-7775 - [Rust] Don't let safe code arbitrarily transmute readers and writersARROW-7777 - [Go] StructBuilder/ListBuilder index out of range panicARROW-7780 - [Release] Fix Windows wheel RC verification script given lack of "m" ABI tag in Python 3.8ARROW-7781 - [C++][Dataset] Filtering on a non-existent column gives a segfaultARROW-7783 - [C++] ARROW_DATASET should enable ARROW_COMPUTEARROW-7785 - [C++] sparse_tensor.cc is extremely slow to compileARROW-7786 - [R] Wire up check_metadata in Table.Equals methodARROW-7789 - [R] Can't initialize arrow objects when R.oo package is loadedARROW-7791 - [C++][Parquet] Fix building error "cannot bind lvalue"ARROW-7792 - [R] read_* functions should close connection to fileARROW-7793 - [Java] If there is a leak the base allocator should release the excess memory to parent before throwing exceptionARROW-7794 - [Rust] cargo publish fails for arrow-flight due to relative path to Flight.protoARROW-7794 - [Rust] cargo publish fails for arrow-flight due to relative path to Flight.protoARROW-7797 - [Release][Rust] Fix arrow-flight's version in datafusion crateARROW-7802 - [C++] Support for LargeBinary and LargeString in the hash kernelARROW-7806 - [Python] Implement to_pandas for lists of LargeBinary/StringARROW-7807 - [R] Installation on RHEL 7 Cannot call io___MemoryMappedFile__Open()ARROW-7809 - [R] vignette does not run on Win 10 nor ubuntuARROW-7813 - [Rust] Fix undefined behaviour and and remove unsafeARROW-7815 - [C++] Fix crashes on corrupt IPC input (OSS-Fuzz)ARROW-7827 - [Python] conda-forge pyarrow package does not have s3 enabledARROW-7832 - [R] Patches to 0.16.0 releaseARROW-7836 - [Rust] "allocate_aligned"/"reallocate" need to initialize memory to avoid UBARROW-7837 - [Java] bug in BaseVariableWidthVector.copyFromSafe results with an index out of bounds exceptionARROW-7838 - [C++] Installed plasma-store-server fails finding BoostARROW-7841 - [C++] HADOOP_HOME doesn't work to find libhdfs.soARROW-7844 - [R] array_to_vector is not thread safeARROW-7848 - Add doc for MapTypeARROW-7852 - [Python] 0.16.0 wheels not compatible with older numpyARROW-7857 - [Python] Failing test with pandas master for extension type conversionARROW-7861 - [C++][Parquet] Add fuzz regression corpus for parquet readerARROW-7884 - [C++][Python] Crash in pq.read_table()ARROW-7887 - [Rust] Filter kernel does not support temporal typesARROW-7889 - [Rust] Datafusion CLI does not support registering Parquet filesARROW-7899 - [Integration][Java] null type integration testARROW-7908 - [R] Can't install package without setting LIBARROW_DOWNLOAD=trueARROW-7922 - [CI][Crossbow] Nightly macOS wheel builds fail (brew bundle edition)ARROW-7923 - [CI][Crossbow] macOS autobrew fails on homebrew-versionsARROW-7926 - [Developer] "archery lint" target is not ergonomic for running a single check like IWYUARROW-7928 - [Python] Example of flight server and client not workingARROW-7931 - [C++] Fix crash on corrupt Map array input (OSS-Fuzz)ARROW-7936 - [Python] FileSystem.from_uri test fails on python 3.5ARROW-7940 - [C++] Unable to generate cmake build with settings other than defaultARROW-7944 - [Python] Test failures without PandasARROW-7956 - [Python] Memory leak in pyarrow functions .ipc.serialize_pandas/deserialize_pandasARROW-7958 - [Java] Update Avro to version 1.9.2ARROW-7962 - [R][Dataset] Followup to "Consolidate Source and Dataset classes"ARROW-7968 - [C++] orc_ep build fails on 64-bit RaspbianARROW-7973 - [Developer][C++] ResourceWarnings in run_cpplint.pyARROW-7974 - [Developer][C++] ResourceWarning in "make check-format"ARROW-7975 - [C++] Do not include padding bytes in "Buffer" IPC metadata accountingARROW-7978 - [Developer] GitHub Actions "lint" task is running include-what-you-use and failingARROW-7980 - [Python] Deserialization with pyarrow fails for certain Timestamp-based data frameARROW-7981 - [C++][Dataset] Fails to compile on gcc 5.4ARROW-7985 - [C++] ListBuilder.Finish fails if underlying value builder is empty and .Reserve'dARROW-7990 - [C++][Developer] Add "archery lint" option for running "iwyu.sh all"ARROW-7992 - [C++] MSVC warning causing Appveyor failure in sort_to_indices.ccARROW-7996 - [Python] Error serializing empty pandas DataFrame with pyarrowARROW-7997 - [Python] Schema equals method with inconsistent docs in pyarrowARROW-7999 - [C++] Fix crash on corrupt Map array input (OSS-Fuzz)ARROW-8000 - [C++] gcc 4.8 build failuresARROW-8003 - [C++] -DBZip2_SOURCE=BUNDLED fails when building with clangARROW-8006 - [C++] Unsafe arrow dictionary recovered from parquetARROW-8007 - [Python] Remove unused and defunct assert_get_object_equal in plasma testsARROW-8008 - [C++/Python] Framework Python is preferred even though not the activated oneARROW-8009 - [Java] Fix the hash code methods for BitVectorARROW-8011 - [C++] Some buffers not resized when reading from ParquetARROW-8013 - [Python][Packaging] Fix manylinux wheelsARROW-8021 - [Python] Appveyor does not appear to be including pandas in test runsARROW-8029 - [R] rstudio/r-base:3.6-centos7 GHA build failing on masterARROW-8036 - [C++] Compilation failure with gtest 1.10.0ARROW-8042 - [Python] pyarrow.ChunkedArray docstring is incorrect regarding zero-length ChunkedArray having no chunksARROW-8057 - [Python] Don't check Schema metadata in __eq__ and __ne__ARROW-8070 - [C++] Cast segfaults on unsupported cast from list to utf8
ARROW-8071 - [GLib] Build error with configureARROW-8075 - [R] Loading R.utils after arrow breaks some arrow functionsARROW-8088 - [C++][Dataset] Partition columns with specified dictionary type result in all nullsARROW-8091 - [CI][Crossbow] Fix nightly homebrew and R failuresARROW-8092 - [CI][Crossbow] OSX wheels fail on bundled bzip2ARROW-8094 - [CI][Crossbow] Nightly valgrind test failsARROW-8095 - [CI][Crossbow] Nightly turbodbc job failsARROW-8098 - [go] Checkptr Failures on Go 1.14ARROW-8099 - [Integration] archery integration --with-LANG flags don't workARROW-8101 - [FlightRPC][Java] Can't read/write only an empty null arrayARROW-8102 - [Dev] Crossbow's version detection doesn't work in the comment bot's scenarioARROW-8105 - [Python] pyarrow.array segfaults when passed masked array with shrunken maskARROW-8106 - [Python] Builds on master broken by pandas 1.0.2 releaseARROW-8110 - [C#] BuildArrays fails if NestedType is includedARROW-8112 - [FlightRPC][C++] Some status codes don't round-trip through gRPCARROW-8119 - [Dev] Make Yaml optional dependency for archeryARROW-8122 - [Python] Empty numpy arrays with shape cannot be deserializedARROW-8125 - [C++] "arrow-tests" target broken with ninja buildARROW-8127 - [C++] [Parquet] Incorrect column chunk metadata for multipage batch writesARROW-8128 - [C#] NestedType children serialized on wrong lengthARROW-8132 - [C++] arrow-s3fs-test failing on masterARROW-8133 - [CI] Github Actions sometimes fail to checkout ArrowARROW-8136 - [C++][Python] Creating dataset from relative path no longer workingARROW-8136 - [C++][Python] Creating dataset from relative path no longer workingARROW-8138 - [C++] parquet::arrow::FileReader cannot read multiple RowGroupARROW-8139 - [C++] FileSystem enum causes attributes warningARROW-8142 - [C++] Casting a chunked array with 0 chunks critical failureARROW-8144 - [CI] Cmake 3.2 nightly build failsARROW-8154 - [Python] HDFS Filesystem does not set environment variables in pyarrow 0.16.0 releaseARROW-8159 - [Python] pyarrow.Schema.from_pandas doesn't support ExtensionDtypeARROW-8166 - [C++] AVX512 intrinsics fail to compile with clang-8 on Ubuntu 18.04ARROW-8176 - [FlightRPC][Integration] Have Flight services bind to port 0 in integrationARROW-8186 - [Python] Dataset expression != returns bool instead of expression for invalid valueARROW-8188 - [R] Adapt to latest checks in R-develARROW-8193 - [C++] arrow-future-test fails to compile on gcc 4.8ARROW-8197 - [Rust] DataFusion "create_physical_plan" returns incorrect schema?ARROW-8206 - [R] Minor fix for backwards compatibility on Linux installationARROW-8209 - [Python] Accessing duplicate column of Table by name gives wrong errorARROW-8213 - [Python][Dataset] Opening a dataset with a local incorrect path gives confusing error messageARROW-8216 - [R][C++][Dataset] Filtering returns all-missing rows where the filtering column is missingARROW-8217 - [R][C++] Fix crashing test in test-dataset.R on 32-bit Windows from ARROW-7979ARROW-8219 - [Rust] sqlparser crate needs to be bumped to version 0.2.5ARROW-8223 - [Python] Schema.from_pandas breaks with pandas nullable integer dtypeARROW-8233 - [CI] Build timeouts on "AMD64 Windows MinGW 64 GLib & Ruby "ARROW-8234 - [CI] Build timeouts on "AMD64 Windows RTools 35"ARROW-8236 - [Rust] Linting GitHub Actions task failingARROW-8237 - [Python] Review Developer build instructions for conda and non-conda usersARROW-8237 - [Python] Review Developer build instructions for conda and non-conda usersARROW-8238 - [C++][Compute] Failed to build compute tests on windows with msvc2015ARROW-8239 - [Java] fix param checks in splitAndTransfer methodARROW-8245 - [Python][Parquet] Skip hidden directories when reading partitioned parquet filesARROW-8254 - [Rust] [DataFusion] CLI is not working as expectedARROW-8255 - [Rust] [DataFusion] COUNT(*) results in confusing errorARROW-8259 - [Rust] [DataFusion] ProjectionPushDownRule does not rewrite LIMITARROW-8268 - [Ruby] Test failure due to lack of built ZSTD supportARROW-8269 - [Python] Failure in "nopandas" build in test_parquet_row_group_fragmentsARROW-8270 - [Python][Flight] Example Flight server with TLS's certificate and key is not workingARROW-8272 - [CI][Python] Test failure on Ubuntu 16.04ARROW-8274 - [C++] Use LZ4 frame format for "LZ4" compression in IPC writeARROW-8276 - [C++][Dataset] Scanning a Fragment does not take into account the partition columnsARROW-8280 - [C++] MinGW builds failing due to CARES-related toolchain issueARROW-8286 - [Python] Creating dataset from pathlib results in UnionDataset instead of FileSystemDatasetARROW-8298 - [C++][CI] MinGW builds fail building grpcARROW-8303 - [Python] Fix test failure caused by non-deterministic dict key ordering on Python 3.5ARROW-8304 - [Flight][Python] Flight client with TLS root certificate is reporting error on do_get()ARROW-8305 - [Java] ExtensionTypeVector should make sure underlyingVector not nullARROW-8310 - [C++] Minio's exceptions not recognized by IsConnectError()ARROW-8315 - [Python][Dataset] Don't rely on ordered dict keys in test_dataset.pyARROW-8323 - [C++] Pin gRPC at v1.27 to avoid compilation error in its headersARROW-8326 - [C++] Don't use deprecated TYPED_TEST_CASEARROW-8327 - [FlightRPC][Java] gRPC trailers may be nullARROW-8331 - [C++] arrow-compute-filter-benchmark fails to compileARROW-8333 - [C++][CI] Always compile benchmarks in some C++ CI entryARROW-8334 - [C++] [Gandiva] Missing DATE32 in LLVM Types / Simple D32 Compute FunctionsARROW-8342 - [Python] dask and kartothek integration tests are failingARROW-8345 - [Python] feather.read_table should not require pandasARROW-8346 - [CI][Ruby] GLib/Ruby macOS build fails on zlibARROW-8349 - [CI][NIGHTLY:gandiva-jar-osx] Use latest pygit2ARROW-8353 - [C++] is_nullable maybe not initialized in parquet writerARROW-8354 - [R] Fix segfault in Table to Array conversionARROW-8357 - [Rust] [DataFusion] Dockerfile for CLI is missing format dirARROW-8358 - [C++] Fix -Wrange-loop-construct warnings in clang-11ARROW-8365 - [C++] Error when writing files to S3 larger than 5 GBARROW-8366 - [Rust] Need to revert recent arrow-flight build changeARROW-8369 - [CI] Fix crossbow wildcard groupsARROW-8373 - [GLib] Problems resolving gobject-introspection, arrow in Meson buildsARROW-8380 - [RUST] StringDictionaryBuilder not publicly exported from arrow::arrayARROW-8384 - [C++][Python] arrow/filesystem/hdfs.h and Python wrapper does not have an option for setting a path to a Kerberos ticketARROW-8386 - [Python] pyarrow.jvm raises error for empty ArraysARROW-8388 - [C++] GCC 4.8 fails to move on returnARROW-8397 - [C++] Fail to compile aggregate_test.cc on Ubuntu 16.04ARROW-8406 - [Python] test_fs fails when run from a different drive on WindowsARROW-8410 - [C++] CMake fails on aarch64 systems that do not support -march=armv8-a+crc+cryptoARROW-8414 - [Python] Non-deterministic row order failure in test_parquet.pyARROW-8414 - [Python] Non-deterministic row order failure in test_parquet.pyARROW-8414 - [Python] Non-deterministic row order failure in test_parquet.pyARROW-8415 - [C++][Packaging] fix gandiva linux jobARROW-8416 - [Python] Provide a "feather" alias in the dataset APIARROW-8420 - [C++] CMake fails to configure on armv7l platform (e.g. Raspberry Pi 3)ARROW-8427 - [C++][Dataset] Do not ignore file paths with underscore/dot when full path was specifiedARROW-8428 - [C++][NIGHTLY:gandiva-jar-trusty] GCC 4.8 failures in C++ unit testsARROW-8429 - [C++] Fix Buffer::CopySlice on 0-sized bufferARROW-8432 - [Python][CI] Failure to download HadoopARROW-8437 - [C++] Remove std::move return value from MakeRandomNullBitmap test utilityARROW-8438 - [C++] arrow-io-memory-benchmark crashesARROW-8439 - [Python] Filesystem docs are outdatedARROW-8441 - [C++] Fix crashes on invalid input (OSS-Fuzz)ARROW-8442 - [Python] NullType.to_pandas_dtype inconsisent with dtype returned in to_pandas/to_numpyARROW-8460 - [Packaging][deb] Ubuntu Focal build is failedARROW-8465 - [Packaging][Python] Windows py35 wheel build fails because of boostARROW-8466 - [Packaging] The python unittests are not running in the windows wheel buildsARROW-8468 - [Document] Fix the incorrect null bits descriptionARROW-8469 - [Dev] Fix nightly docker tests on azureARROW-8478 - [Java] Rollback contrib package changes.ARROW-8498 - [Python] Schema.from_pandas fails on extension type, while Table.from_pandas worksPARQUET-1780 - [C++] Set ColumnMetadata.encoding_stats fieldPARQUET-1788 - [C++] ColumnWriter has undefined behavior when writing arrow chunksPARQUET-1797 - [C++] Fix fuzzing errorsPARQUET-1799 - [C++] Stream API: Relax schema checking when readingPARQUET-1810 - [C++] Fix undefined behaviour on invalid enum values (OSS-Fuzz)PARQUET-1813 - [C++] Remove logging statement in unit testPARQUET-1819 - [C++] Fix crashes on corrupt IPC input (OSS-Fuzz)PARQUET-1819 - [C++] Fix crashes on corrupt IPC input (OSS-Fuzz)PARQUET-1823 - [C++] Invalid RowGroup returned when reading with parquet::arrow::FileReader->RowGroup(i)->Column(j)PARQUET-1824 - [C++] Fix crashes on invalid input (OSS-Fuzz)PARQUET-1829 - [C++] Fix crashes on invalid input (OSS-Fuzz)PARQUET-1831 - [C++] Fix crashes on invalid input (OSS-Fuzz)PARQUET-1835 - [C++] Fix crashes on invalid input (OSS-Fuzz)New Features and Improvements
ARROW-590 - [Integration] Add integration tests for Union typesARROW-1470 - [C++] Add BufferAllocator abstract interfaceARROW-1560 - [C++] Kernel implementations for "match" functionARROW-1571 - [C++] Implement argsort kernels (sort indices) for integers using O(n) counting sortARROW-1581 - [Packaging] Tooling to make nightly wheels available for installARROW-1582 - [Python] Set up + document nightly conda builds for macOSARROW-1636 - [Format] Integration tests for null typeARROW-2447 - [C++] Create a device abstractionARROW-2882 - [C++][Python] Support AWS Firehose partition_scheme implementation for Parquet datasetsARROW-3054 - [Packaging] Tooling to enable nightly conda packages to be updated to some anaconda.org channelARROW-3410 - [C++][Dataset] Streaming CSV reader interface for memory-constrainted environmentsARROW-3750 - [R] Pass various wrapped Arrow objects created in Python into R with zero copy via reticulateARROW-4120 - [Python] Define process for testing procedures that check for no macro-level memory leaksARROW-4226 - [Format][C++] Add CSF sparse tensor supportARROW-4286 - [C++/R] Namespace vendored BoostARROW-4304 - [Rust] Enhance documentation for arrowARROW-4428 - [R] Feature flags for R buildARROW-4482 - [Website] Add blog archive pageARROW-4815 - [Rust] [DataFusion] Add support for * in SQL projectionARROW-5357 - [Rust] Add capacity field in BufferARROW-5405 - [Documentation] Move integration testing documentation to Sphinx docs, add instructions for JavaScriptARROW-5497 - [Release] Build and publish R/Java/JS docsARROW-5501 - [R] Reorganize read/write file/stream functionsARROW-5510 - [Format] Feather V2 based on Arrow IPC file format, with compression supportARROW-5563 - [Format] Update integration test JSON format documentationARROW-5585 - [Go] rename arrow.TypeEquals into arrow.TypeEqualARROW-5742 - [CI] Add daily / weekly Valgrind buildARROW-5757 - [Python] Stop supporting Python 2.7ARROW-5949 - [Rust] Implement DictionaryArrayARROW-6165 - [Integration] Use multiprocessing to run integration tests on multiple CPU coresARROW-6176 - [Python] Allow to subclass ExtensionArray to attach to custom extension typeARROW-6275 - [C++] Deprecate RecordBatchReader::ReadNextARROW-6393 - [C++] Add EqualOptions support in SparseTensor::EqualsARROW-6479 - [C++] inline errors from external projects' build logsARROW-6510 - [Python][Filesystem] Expose nanosecond resolution mtimeARROW-6666 - [Rust] [DataFusion] Implement string literal expressionARROW-6724 - [C++] Add simpler static ctor for BufferOutputStream than the current Create functionARROW-6821 - [C++][Parquet] Do not require Thrift compiler when building (but still require library)ARROW-6823 - [C++][Python][R] Support metadata in the feather format?ARROW-6829 - [Docs] Migrate integration test docs to Sphinx, fix instructions after ARROW-6466ARROW-6837 - [C++/Python] access File Footer custom_metadataARROW-6841 - [C++] Upgrade to LLVM 8ARROW-6875 - [FlightRPC] Implement Criteria for ListFlights RPC / list_flights methodARROW-6915 - [Developer] Do not overwrite minor release version with merge script, even if not specified by committerARROW-6947 - [Rust] [DataFusion] Add support for scalar UDFsARROW-6996 - [Python] Expose boolean filter kernel on TableARROW-7044 - [Release] Create a post release script for the home-brew formulasARROW-7048 - [Java] Support for combining multiple vectors under VectorSchemaRootARROW-7063 - [C++] Schema print method prints too much metadataARROW-7073 - [Java] Support concating vectors values in batchARROW-7080 - [Python][Parquet][C++] Expose parquet field_id in Schema objectsARROW-7091 - [C++] Move all factories to type_fwd.hARROW-7119 - [C++][CI] Use scripts/util_coredump.sh to show automatic backtracesARROW-7201 - [GLib][Gandiva] Add support for BooleanNodeARROW-7202 - [R][CI] Improve rwinlib building on CI to stop re-downloading dependenciesARROW-7222 - [Python][Release] Wipe any existing generated Python API documentation when updating websiteARROW-7233 - [C++] Add Result APIs to IPC module
ARROW-7256 - [C++] Remove ARROW_MEMORY_POOL_DEFAULT macroARROW-7330 - [C++] Add Result to APIs to arrow/gpu
ARROW-7332 - [C++][Parquet] Explicitly catch status exceptions in PARQUET_CATCH_NOT_OKARROW-7336 - [C++] Implement MinMax options to not skip nullsARROW-7338 - [C++] Improve InMemoryDataSource to support generator instead of static listARROW-7365 - [Python] Support FixedSizeList type in conversion to numpy/pandasARROW-7373 - [C++][Dataset] Remove FileSourceARROW-7400 - [Java] Avoids the worst case for quick sortARROW-7412 - [C++][Dataset] Ensure that dataset code is robust to schemas with duplicate field namesARROW-7419 - [Python] Support SparseCSCMatrixARROW-7427 - [Python] Support SparseCSFTensorARROW-7428 - [Format][C++] Add serialization for CSF sparse tensorsARROW-7444 - [GLib] Add LocalFileSystem supportARROW-7462 - [C++] Add CpuInfo detection for Arm64 ArchitectureARROW-7491 - [Java] Improve the performance of aligningARROW-7499 - [C++] CMake should collect libs when making static buildARROW-7501 - [C++] CMake build_thrift should build flex and bison if necessaryARROW-7515 - [C++] Rename nonexistent and non_existent to not_foundARROW-7524 - [C++][CI] Build parquet support in the VS2019 GitHub Actions jobARROW-7530 - [Developer] Do not include list of commits from PR in squashed summary messageARROW-7534 - [Java] Create a new java/contrib moduleARROW-7547 - [C++] [Python] [Dataset] Additional reader options in ParquetFileFormatARROW-7555 - [Python] Drop support for python 2.7ARROW-7587 - [C++][Compute] Add Top-k kernelARROW-7608 - [C++][Dataset] Expose more informational propertiesARROW-7615 - [CI][Gandiva] Ensure that the gandiva_jni library has only a whitelisted set of shared dependencies as part of Travis CI jobARROW-7616 - [Java] Support comparing value ranges for dense union vectorARROW-7625 - [GLib] Parquet GLib and Red Parquet (Ruby) do not allow specifying compression typeARROW-7641 - [R] Make dataset vignette have executable codeARROW-7662 - [R] Support creating ListArray from R listARROW-7664 - [C++] Extract localfs default from FileSystemFromUriARROW-7675 - [R][CI] Move Windows CI from Appveyor to GHAARROW-7679 - [R] Cleaner interface for creating UnionDatasetARROW-7684 - [Rust] Provide example of Flight server for DataFusionARROW-7685 - [Developer] Add support for GitHub Actions to CrossbowARROW-7691 - [C++] Verify missing fields when walking Flatbuffers dataARROW-7708 - [Release] Include PARQUET commits from git changelog in release changelogsARROW-7712 - [CI][Crossbow] Fix or delete fuzzit jobsARROW-7720 - [C++][Python] Add check_metadata argument to Table.equalsARROW-7725 - [C++] Add infrastructure for unity builds and precompiled headersARROW-7726 - [CI] [C++] Use boost binaries on Windows GHA buildARROW-7729 - [Python][CI] Pin pandas version to 0.25 in the dask integration testARROW-7733 - [Developer] Install locally a new enough version of Go for release verification scriptARROW-7735 - [Release] conda-forge channel is missing for verifying wheelsARROW-7736 - [Release] Binary verification sometimes fails with transient errorARROW-7739 - [GLib] Use placement new to initialize shared_ptr object in private structsARROW-7741 - [C++][Parquet] Incorporate new level generation logic in parquet write path with a flag to revert back to old logicARROW-7742 - [GLib] Add support for MapArrayARROW-7745 - [Doc] [C++] Update Parquet documentationARROW-7749 - [C++] Link some more tests togetherARROW-7750 - [Release] Make the source release verification script restartableARROW-7751 - [Release] macOS wheel verification also needs arrow-testingARROW-7752 - [Release] Enable and test dataset in the verification scriptARROW-7754 - [C++] Result is slow
ARROW-7761 - [C++] Add S3 support to fs::FileSystemFromUriARROW-7764 - [C++] Builders allocate a null bitmap buffer even if there is no nullsARROW-7771 - [Developer] Use ARROW_TMPDIR environment variable in the verification scripts instead of TMPDIRARROW-7774 - [Packaging][Python] Update macos and windows wheel filenamesARROW-7787 - [Rust] Add collect to Table APIARROW-7788 - [C++] Add schema conversion support for map typeARROW-7790 - [Website] Update how to install Linux packagesARROW-7795 - [Rust - DataFusion] Support boolean negation (NOT)ARROW-7796 - [R] write_* functions should invisibly return their inputsARROW-7799 - [R][CI] Remove flatbuffers from homebrew formulaeARROW-7804 - [C++][R] Compile error on macOS 10.11ARROW-7812 - [Packaging][Python] Upgrade LLVM in manylinux1 docker imageARROW-7817 - [CI] macOS R autobrew nightly failed on installing dependency from sourceARROW-7819 - [C++][Gandiva] Add DumpIR to Filter/Projector classesARROW-7824 - [C++][Dataset] Provide Dataset writing to IPC formatARROW-7828 - [Release] Remove SSH keys for internal useARROW-7829 - [R] Test R bindings on clangARROW-7833 - [R] Make install_arrow() actually install arrowARROW-7834 - [Release] Post release task for updating the documentationsARROW-7839 - [Python][Dataset] Add IPC format to python bindingsARROW-7846 - [Python][Dev] Remove last dependencies on sixARROW-7847 - [Website] Write a blog post about fuzzingARROW-7849 - [Packaging][Python] Remove the remaining py27 crossbow wheel tasks from the nightliesARROW-7858 - [C++][Python] Support casting an Extension type to its storage typeARROW-7859 - [R] Minor patches for CRAN submission 0.16.0.2ARROW-7860 - [C++] Support cast to/from halffloatARROW-7862 - [R] Linux installation should run quieter by defaultARROW-7863 - [C++][Python][CI] Ensure running HDFS related testsARROW-7864 - [R] Make sure bundled installation works even if there are system packagesARROW-7865 - [R] Test builds on latest Linux versionsARROW-7868 - [Crossbow] Reduce GitHub API query parallelismARROW-7869 - [Python] Boost::system and boost::filesystem not necessary anymore in Python wheelsARROW-7872 - [Python] Support conversion of list-of-struct in Array/Table.to_pandasARROW-7874 - [Python][Archery] Validate docstrings with numpydocARROW-7876 - [R] Installation fails in the documentation generation imageARROW-7877 - [Packaging] Fix crossbow deployment to github artifactsARROW-7879 - [C++][Doc] Add doc for the Device APIARROW-7880 - [CI][R] R sanitizer job is not really workingARROW-7881 - [C++] Fix pedantic warningsARROW-7882 - [C++][Gandiva] Optimise like function for substring patternARROW-7886 - [C++][Dataset] Consolidate Source and DatasetARROW-7888 - [Python] Allow using a more modern version of jpype in pyarrow.jvmARROW-7890 - [C++] Add Promise / Future implementationARROW-7891 - [C++] RecordBatch->Equals should also have a check_metadata argumentARROW-7892 - [Python] Expose FilesystemSource.format attributeARROW-7895 - [Python] Remove more python 2.7 cruftARROW-7896 - [C++] Refactor from #include guards to #pragma onceARROW-7897 - [Packaging] Temporarily disable artifact uploading until we fix the deployment issuesARROW-7898 - [Python] Reduce the number docstring violations using numpydocARROW-7904 - [C++] Decide about Field/Schema metadata printing parameters and how much to show by defaultARROW-7907 - [Python] Conversion to pandas of empty table with timestamp type abortsARROW-7912 - [Format] C data interfaceARROW-7913 - [C++][Python][R] C++ implementation of C data interfaceARROW-7915 - [CI] [Python] Run tests with Python development mode enabledARROW-7916 - [C++][Dataset] Project IPC record batches to materialized fieldsARROW-7917 - [CMake] FindPythonInterp should check for python3ARROW-7919 - [R] install_arrow() should conda install if appropriateARROW-7920 - [R] Fill in some missing input validationARROW-7921 - [Go] Add Reset method to various components and clean up commentsARROW-7927 - [C++] Fix 'cpu_info.cc' compilation warningARROW-7929 - [C++] CMake target names differ from upstream provided namesARROW-7930 - [Python][CI] Test jpype integration in CIARROW-7932 - [Rust] [Parquet] Implement array reader for temporal typesARROW-7934 - [C++] Fix UriEscape for empty stringARROW-7935 - [Java] Remove Netty dependency for BufferAllocator and ReferenceManagerARROW-7937 - [Python][Packaging] Remove boost from the macos wheelsARROW-7941 - [Rust] [DataFusion] Logical plan should support unresolved column referencesARROW-7943 - [C++][Parquet] Add a new level builder capable of handling nested dataARROW-7947 - [Rust] [Flight] [DataFusion] Implement example for get_schemaARROW-7949 - [Developer] Update to '.gitignore' to not track user specific 'cpp/Brewfile.lock.json' fileARROW-7951 - [Python][Parquet] Expose BYTE_STREAM_SPLIT to pyarrowARROW-7959 - [Ruby] Add support for Ruby 2.3 againARROW-7963 - [C++][Python][Dataset] Expose listing fragmentsARROW-7965 - [Python] Refine higher level dataset APIARROW-7966 - [Integration][Flight][C++] Client should verify each batch independentlyARROW-7969 - [Packaging] Use cURL to upload artifactsARROW-7970 - [Packaging][Python] Use system boost to build the macos wheelsARROW-7971 - [Rust] Create rowcount utilityARROW-7977 - [C++] Rename fs::FileStats to fs::FileInfoARROW-7979 - [C++] Implement experimental buffer compression in IPC messagesARROW-7982 - [C++] Let ArrayDataVisitor accept void-returning functionsARROW-7983 - [CI][R] Nightly builds should be more verbose when they failARROW-7984 - [R] Check for valid inputs in more placesARROW-7986 - [Python] pa.Array.from_pandas cannot convert pandas.Series containing pyspark.ml.linalg.SparseVectorARROW-7987 - [CI][R] Fix for verbose nightly buildsARROW-7988 - [R] Fix on.exit calls in reticulate bindingsARROW-7991 - [C++][Plasma] Allow option for evicting if full when creating an objectARROW-7993 - [Java] Support decimal type in ComplexCopierARROW-7994 - [CI][C++] Move AppVeyor MinGW builds to GitHub ActionsARROW-7995 - [C++] IO: coalescing and caching read rangesARROW-7998 - [C++][Plasma] Make Seal requests synchronousARROW-8005 - [Website] Review and adjust any usages of Apache dist system from website / toolsARROW-8014 - [C++] Provide CMake targets to test only within a given labelARROW-8016 - [Developer] Fix deprecation warning in PR merge toolARROW-8018 - [C++][Parquet]Parquet Modular EncryptionARROW-8024 - [R] Bindings for BinaryType and FixedBinaryTypeARROW-8026 - [Python] Support memoryview in addition to string value types for constructing string and binary type arraysARROW-8027 - [Developer][Integration] Add integration tests for duplicate field namesARROW-8028 - [Go] Allow duplicate field names in schemas and nested typesARROW-8030 - [C++][Plasma] Fix inconsistent comment styleARROW-8035 - [Developer][Integration] Add integration tests for extension typesARROW-8039 - [Python][Dataset] Support using dataset API in pyarrow.parquet with a minimal ParquetDataset shimARROW-8044 - [CI][NIGHTLY:gandiva-jar-osx] pygit2 needs libgit2 v1.0.xARROW-8055 - [GLib][Ruby] Add some metadata bindings to GArrowSchemaARROW-8058 - [C++][Python][Dataset] Provide an option to toggle validation and schema inference in FileSystemDatasetFactoryOptionsARROW-8059 - [Python] Make FileSystem objects serializableARROW-8060 - [Python] Make dataset Expression objects serializableARROW-8061 - [C++][Dataset] Ability to specify granularity of ParquetFileFragment (support row groups)ARROW-8063 - [Python] Add user guide documentation for Datasets APIARROW-8064 - [Dev] Implement Comment bot via Github actionsARROW-8069 - [C++] Should the default value of "check_metadata" arguments of Equals methods be "true"?ARROW-8072 - [C++][Plasma] Add const constraint when parsing dataARROW-8077 - [Python] Add wheel build script and Crossbow configuration for Windows on Python 3.5ARROW-8079 - [Python] Implement a wrapper for KeyValueMetadata, duck-typing dict where relevantARROW-8080 - [C++] Add AVX512 build optionARROW-8082 - [Java][Plasma] Add JNI list() interfaceARROW-8083 - [GLib] Add support for Peek() to GIOInputStreamARROW-8086 - [Java] Support writing decimal from big endian byte array in UnionListWriterARROW-8087 - [C++][Dataset] Order of keys with HivePartitioning is lost in resulting schemaARROW-8096 - [C++][Gandiva] Create null node of Interval typeARROW-8097 - [Dev] Comment bot's crossbow command acts on the master branchARROW-8103 - [R] Make default Linux build more minimalARROW-8104 - [C++] Don't install bundled ThriftARROW-8107 - [Packaging][APT] Use HTTPS for LLVM APT repository for Debian GNU/Linux stretchARROW-8109 - [Packaging][APT] Drop support for Ubuntu DiscoARROW-8117 - [Rust] [Datafusion] Allow CAST from number to timestampARROW-8118 - [R] dim method for FileSystemDatasetARROW-8120 - [Packaging][APT] Add support for Ubuntu FocalARROW-8123 - [Rust] [DataFusion] Create LogicalPlanBuilderARROW-8124 - [Rust] Update library dependenciesARROW-8126 - [C++][Compute] Add Top-K kernel benchmarkARROW-8129 - [C++][Compute] Refine compare sorting kernelARROW-8130 - [C++][Gandiva] Fix Dex visitor in llvm_generator to handle interval typeARROW-8140 - [Developer] Follow NullType -> NullField changeARROW-8141 - [C++] Optimize BM_PlainDecodingBoolean performance using AVX512 Intrinsics APIARROW-8145 - [C++] Rename GetTargetInfosARROW-8146 - [C++] Add per-filesystem facility to sanitize a pathARROW-8150 - [Rust] Allow writing custom FileMetaData k/v pairsARROW-8151 - [Benchmarking][Dataset] Benchmark Parquet read performance with S3FileARROW-8153 - [Packaging] Update the conda feedstock files and upload artifacts to AnacondaARROW-8158 - [Java] Getting length of data buffer and base variable width vectorARROW-8164 - [C++][Dataset] Let datasets be viewable with non-identical schemaARROW-8165 - [Packaging] Make nightly wheels available on a PyPI serverARROW-8167 - [CI] Add support for skipping builds with skip pattern in pull request titleARROW-8168 - [Java][Plasma] Improve Java Plasma client off-heap memory usageARROW-8177 - [Rust] Make schema_to_fb_offset publicARROW-8178 - [C++] Upgrade to Flatbuffers 1.12ARROW-8179 - [R] Windows build script tweaking for nightly packaging on GHAARROW-8181 - [Java][FlightRPC] Expose transport error metadataARROW-8182 - [Packaging] Increment the version number detected from the latest git tagARROW-8183 - [c++][FlightRPC] Expose transport error metadataARROW-8184 - [Packaging] Use arrow-nightlies organization name on Anaconda and Gemfury to host the nightliesARROW-8185 - [Packaging] Document the available nightly wheels and conda packagesARROW-8187 - [R] Make test assertions robust to i18nARROW-8191 - [Packaging][APT] Fix cmake removal in Debian GNU/Linux StretchARROW-8192 - [C++] script for unpack avx512 intrinsics codeARROW-8194 - [CI] Github Actions Windows job should run tests in parallelARROW-8195 - [CI] Remove Boost download step in Github ActionsARROW-8198 - [C++] Diffing should handle null arraysARROW-8200 - [GLib] Rename garrow_file_system_target_info{,s}() to ..._file_info{,s}()ARROW-8203 - [C#] "dotnet pack" is failedARROW-8204 - [Rust] [DataFusion] Add support for aliased expressions in SQLARROW-8207 - [Packaging][wheel] Use LLVM 8 in manylinux2010 and manylinux2014ARROW-8215 - [CI][GLib] Meson install fails in the macOS buildARROW-8218 - [C++] Parallelize decompression at field level in experimental IPC compression codeARROW-8220 - [Python] Make dataset FileFormat objects serializableARROW-8222 - [C++] Use bcp to make a slim boost for bundled buildARROW-8224 - [C++] Remove APIs deprecated prior to 0.16.0ARROW-8225 - [Rust] IPC reader must respect continuation markersARROW-8225 - [Rust] IPC reader must respect continuation markersARROW-8227 - [C++] Refine SIMD feature definitionsARROW-8231 - [Rust] Parse key_value_metadata from parquet FileMetaData into arrow schema metadataARROW-8232 - [Python] Deprecate pa.open_file and pa.open_stream in favor of pa.ipc.open_file/open_streamARROW-8235 - [C++][Compute] Filter out nulls by defaultARROW-8241 - [Rust] Add convenience methods to SchemaARROW-8242 - [C++] Flight fails to compile on GCC 4.8ARROW-8243 - [Rust] [DataFusion] Fix inconsistent API in LogicalPlanBuilderARROW-8244 - [Python][Parquet] Add `write_to_dataset` option to populate the "file_path" metadata fieldsARROW-8246 - [C++] Add -Wa,-mbig-obj when compiling with MinGW to avoid linking errorsARROW-8247 - [Python] Expose Parquet writing "engine" setting in pyarrow.parquet.write_tableARROW-8249 - [Rust] [DataFusion] Make Table and LogicalPlanBuilder APIs more consistentARROW-8252 - [CI][Ruby] Add Ubuntu 20.04ARROW-8256 - [Rust] [DataFusion] Update CLI documentation for 0.17.0 releaseARROW-8264 - [Rust] [DataFusion] Create utility for printing record batchesARROW-8266 - [C++] Add backup mirrors for external project source downloadsARROW-8267 - [CI][GLib] Failed to build on Ubuntu 16.04ARROW-8271 - [Packaging] Allow wheel upload failures to gemfuryARROW-8275 - [Python][Docs] Review Feather + IPC file documentation per "Feather V2" changesARROW-8277 - [Python] RecordBatch interface improvementsARROW-8279 - [C++] Do not export symbols from Codec implementations, remove need for PIMPL patternARROW-8288 - [Python] Expose with_ modifiers on DataTypeARROW-8290 - [Python][Dataset] Improve ergonomy of the FileSystemDataset constructorARROW-8291 - [Packaging] Conda nightly builds can't locate NumpyARROW-8292 - [Python][Dataset] Passthrough schema to Factory.finish() in dataset() functionARROW-8294 - [Format][Flight] Add DoExchange RPC to Flight protocolARROW-8295 - [C++][Dataset] IpcFileFormat should expliclity push down column projectionARROW-8299 - [C++] Reusable "optional ParallelFor" function for optional use of multithreadingARROW-8300 - [R] Documentation and changelog updates for 0.17ARROW-8307 - [Python] Expose use_memory_map option in pyarrow.feather APIsARROW-8308 - [Rust] [Flight] Implement DoExchange on examplesARROW-8309 - [CI] C++/Java/Rust workflows should trigger on changes to Flight.protoARROW-8311 - [C++] Add push style stream format readerARROW-8316 - [CI] Set docker-compose to use docker-cli instead of docker-py for building imagesARROW-8319 - [CI] Install thrift compiler in the debian buildARROW-8320 - [Documentation][Format] Clarify (lack of) alignment requirements in C data interfaceARROW-8321 - [CI] Use bundled thrift in Fedora 30 buildARROW-8322 - [CI] Fix C# workflow file syntaxARROW-8325 - [R][CI] Stop including boost in R windows bundleARROW-8329 - [Documentation][C++] Undocumented FilterOptions argument in Filter kernelARROW-8330 - [Documentation] The post release script generates the documentation with a development versionARROW-8332 - [C++] Require Thrift compiler to use system libthrift for Parquet buildARROW-8335 - [Release] Add crossbow jobs to run release verificationARROW-8336 - [Packaging][deb] Use libthrift-dev on Debian 10 and Ubuntu 19.10 or laterARROW-8341 - [Packaging][deb] Fail to build by no disk spaceARROW-8343 - [GLib] Add GArrowRecordBatchIteratorARROW-8347 - [C++] Add Result APIs to Array methods
ARROW-8351 - [R][CI] Store the Rtools-built Arrow C++ library as a build artifactARROW-8352 - [R] Add install_pyarrow()ARROW-8356 - [Developer] Support * wildcards with "crossbow submit" via GitHub actionsARROW-8361 - [C++] Add Result APIs to Buffer methods and functions
ARROW-8362 - [Crossbow] Ensure that the locally generated version is used in the docker tasksARROW-8367 - [C++] Deprecate Buffer::FromString(..., pool)ARROW-8368 - [Format] In C interface, clarify resource management for consumers needing only a subset of child fields in ArrowArrayARROW-8370 - [C++] Add Result to type / schema APIs
ARROW-8371 - [Crossbow] Implement and exercise sanity checks for tasks.ymlARROW-8372 - [C++] Add Result to table / record batch APIs
ARROW-8375 - [CI][R] Make Windows tests more verbose in case of segfaultARROW-8376 - [R] Add experimental interface to ScanTask/RecordBatch iteratorsARROW-8387 - [Rust] Make schema_to_fb publicARROW-8389 - [Integration] Run tests in parallelARROW-8390 - [R] Expose schema unification featuresARROW-8393 - [C++][Gandiva] Make gandiva function registry case-insensitiveARROW-8396 - [Rust] Remove libc from dependenciesARROW-8398 - [Python] Remove deprecation warnings originating from python testsARROW-8401 - [C++] Add AVX2/AVX512 version of ByteStreamSplitDecode/ByteStreamSplitEncodeARROW-8403 - [C++] Add ToString() to ChunkedArray, Table and RecordBatchARROW-8407 - [Rust] Add rustdoc for Dictionary typeARROW-8408 - [Python] Add memory_map= toggle to pyarrow.feather.read_featherARROW-8409 - [R] Add arrow::cpu_count, arrow::set_cpu_count wrapper functions a la PythonARROW-8412 - [C++][Gandiva] Fix gandiva date_diff function definitionsARROW-8433 - [R] Add feather alias for ipc format in dataset APIARROW-8444 - [Documentation] Fix spelling errors across the codebaseARROW-8449 - [R] Use CMAKE_UNITY_BUILD everywhereARROW-8450 - [Integration][C++] Implement large list/binary/utf8 integrationARROW-8457 - [C++] bridge test does not take care of endiannessARROW-8458 - [C++] Prefer the original mirrors for the bundled thirdparty dependenciesARROW-8461 - [Packaging][deb] Use zstd package for Ubuntu XenialARROW-8463 - [CI] Balance the nightly test builds between CircleCI, Azure and GithubARROW-8679 - [Python] supporting pandas sparse series in pyarrowPARQUET-458 - [C++] Implement support for DataPageV2PARQUET-1663 - [C++] Provide API to check the presence of complex data typesPARQUET-1716 - [C++] Add support for BYTE_STREAM_SPLIT encodingPARQUET-1770 - [C++][CI] Add fuzz target for reading Parquet filesPARQUET-1785 - [C++] Improve code reusability in encoding-test.ccPARQUET-1786 - [C++] Use simd to improve BYTE_STREAM_SPLIT decoding performancePARQUET-1806 - [C++] [CI] Improve fuzzing seed corpusPARQUET-1825 - [C++] Fix compilation error in column_io_benchmark.ccPARQUET-1828 - [C++] Add a SSE2 path for the ByteStreamSplit encoder implementationPARQUET-1840 - [C++] DecodeSpaced copies more values then necessaryApache Arrow 0.16.0 (2020-02-07)
Bug Fixes
ARROW-3783 - [R] Incorrect collection of float typeARROW-3962 - [Go] Support null values while reading a CSV file.ARROW-4470 - [Python] Pyarrow using considerable more memory when reading partitioned Parquet fileARROW-4998 - [R] R package fails to install on OSXARROW-5575 - [C++] arrowConfig.cmake includes uninstalled targetsARROW-5655 - [Python] Table.from_pydict/from_arrays not using types in specified schema correctlyARROW-5680 - [Rust] datafusion group-by tests depends on result set orderARROW-6157 - [Python][C++] UnionArray with invalid data passes validation / leads to segfaultsARROW-6195 - [C++] CMake fails with file not found error while bundling thrift if python is not installedARROW-6298 - [Rust] [CI] Examples are not being tested in CIARROW-6320 - [C++] Arrow utilities are linked staticallyARROW-6429 - [CI][Crossbow] Nightly spark integration job failsARROW-6445 - [CI][Crossbow] Nightly Gandiva jar trusty job failsARROW-6567 - [Rust] [DataFusion] SQL aggregate query execution assume grouping expressions precede aggregate expressionsARROW-6581 - [C++] Fix fuzzit job submissionARROW-6704 - [C++] Cast from timestamp to higher resolution does not check out of bounds timestampsARROW-6708 - [C++] "cannot find -lboost_filesystem_static"ARROW-6728 - [C#] Support reading and writing Date32 and Date64 arraysARROW-6736 - [Rust] [DataFusion] Aggregate expressions get evaluated repeatedlyARROW-6740 - [Python] Unable to delete closed MemoryMappedFile on WindowsARROW-6745 - [Rust] Fix a variety of typosARROW-6749 - [Python] Conversion of non-ns timestamp array to numpy gives wrong valuesARROW-6750 - [Python] Silence S3 error logs by defaultARROW-6761 - [Rust] Travis CI builds not respecting rust-toolchainARROW-6762 - [C++] JSON reader segfaults on newlineARROW-6785 - [JS] Remove superfluous child assignmentARROW-6786 - [C++] arrow-dataset-file-parquet-test is slowARROW-6795 - [C#] Reading large Arrow files in C# results in an exceptionARROW-6798 - [CI] [Rust] Improve build times by caching dependencies in the Docker imageARROW-6801 - [Rust] Arrow source release tarball is missing benchmarksARROW-6806 - [C++] Segfault deserializing ListArray containing null/empty listARROW-6808 - [Ruby] Ensure requiring suitable MSYS2 packageARROW-6809 - [RUBY] Gem does not install on macOS due to glib2 3.3.7 compilation failureARROW-6812 - [Java] Remove Dremio Corp. from License HeaderARROW-6813 - [Ruby] Arrow::Table.load with headers=true leads to exception in Arrow 0.15ARROW-6820 - [C++] [Doc] [Format] Map specification and implementation inconsistentARROW-6834 - [C++] Pin gtest to 1.8.1 to triage failing Appveyor / MSVC buildARROW-6835 - [Archery][CMake] Restore ARROW_LINT_ONLYARROW-6842 - [Website] Jekyll error building websiteARROW-6844 - [C++][Parquet][Python] List columns read broken with 0.15.0
ARROW-6846 - [C++] Build failures with glog enabledARROW-6857 - [Python][C++] Segfault for dictionary_encode on empty chunked_array (edge case)ARROW-6859 - [CI][Nightly] Disable docker layer caching for CircleCI tasksARROW-6860 - [Python] Only link libarrow_flight.so to pyarrow._flightARROW-6861 - [Python] arrow-0.15.0 reading arrow-0.14.1-output Parquet dictionary column: Failure reading column: IOError: Arrow error: Invalid: Resize cannot downsizeARROW-6864 - [C++] bz2 / zstd tests not enabledARROW-6867 - [FlightRPC][Java] Flight server can hang JVM on shutdownARROW-6868 - [Go] slicing Struct array does not slice child fieldsARROW-6869 - [C++] Dictionary "delta" building logic in builder_dict.h produces invalid arraysARROW-6873 - [Python] Stale CColumn reference break Cython cimport pyarrowARROW-6874 - [Python] Memory leak in Table.to_pandas() when conversion to object dtypeARROW-6876 - [Python] Reading parquet file with many columns becomes slow for 0.15.0ARROW-6877 - [C++] Boost not found from the correct environmentARROW-6878 - [Python] pa.array() does not handle list of dicts with bytes keys correctly under python3ARROW-6882 - [Python] cannot create a chunked_array from dictionary_encoding resultARROW-6885 - [Python] Remove superfluous skipped timedelta testARROW-6886 - [C++] arrow::io header nvcc compiler warningsARROW-6898 - [Java] Fix potential memory leak in ArrowWriter and several test classesARROW-6898 - [Java] Fix potential memory leak in ArrowWriter and several test classesARROW-6899 - [Python] to_pandas() not implemented on list
ARROW-6901 - [Rust][Parquet] SerializedFileWriter writes total_num_rows as zeroARROW-6903 - [Python] Wheels broken after ARROW-6860 changesARROW-6905 - [Packaging][OSX] Nightly builds on MacOS are failing because of brew compile timeoutsARROW-6910 - [Python] pyarrow.parquet.read_table(...) takes up lots of memory which is not released until program exitsARROW-6913 - [R] Potential bug in compute.ccARROW-6914 - [CI] docker-clang-format nightly failingARROW-6922 - [Python] Pandas master build is failing (MultiIndex.levels change)ARROW-6925 - [C++] Arrow fails to buld on MacOS 10.13.6 using brew gcc 7 and 8ARROW-6929 - [C++] ValidateArray is out of sync with the ListArray IPC specificationARROW-6937 - [Packaging][Python] Fix conda linux and OSX wheel nightly buildsARROW-6938 - [Python] Windows wheel depends on zstd.dll and libbz2.dll, which are not bundledARROW-6948 - [Rust] [Parquet] Fix bool array support in arrow reader.ARROW-6950 - [C++][Dataset] Add example/benchmark for reading parquet files with datasetARROW-6957 - [CI][Crossbow] Nightly R with sanitizers build fails installing dependenciesARROW-6962 - [C++] [CI] Stop compiling with -WeverythingARROW-6966 - [Go] 32bit memset is nullARROW-6977 - [C++] Only enable jemalloc background_thread if feature is supportedARROW-6983 - [C++] Threaded task group crashes sometimesARROW-6989 - [Python][C++] Assert is triggered when decimal type inference occurs on a value with out of range precisionARROW-6992 - [C++]: Undefined Behavior sanitizer build option fails with GCCARROW-6999 - [Python] KeyError: '__index_level_0__' passing Table.from_pandas its own schemaARROW-7013 - [C++] arrow-dataset pkgconfig is incompleteARROW-7020 - [Java] Fix the bugs when calculating vector hash codeARROW-7021 - [Java] UnionFixedSizeListWriter decimal type should check writer indexARROW-7022 - [Python] __arrow_array__ does not work for ExtensionTypes in Table.from_pandasARROW-7023 - [Python] pa.array does not use "from_pandas" semantics for pd.IndexARROW-7024 - [CI][R] Update R dependencies for Conda buildARROW-7027 - [Python] pa.table(..) returns instead of raises error if passing invalid objectARROW-7033 - [C++] Error in./configure step for jemalloc when building on OSX 10.14.6ARROW-7045 - [R] Factor type not preserved in Parquet roundtripARROW-7050 - [R] Fix compiler warnings in R bindingsARROW-7053 - [Python] setuptools-scm produces incorrect version at apache-arrow-0.15.1 tagARROW-7056 - [Python] Test errors without S3ARROW-7059 - [Python] Reading parquet file with many columns is much slower in 0.15.x versus 0.14.xARROW-7074 - [C++] ASSERT_OK_AND_ASSIGN crashes when failingARROW-7077 - [C++] Unsupported Dict->T cast crashes instead of returning errorARROW-7087 - [Python] Table Metadata disappear when we write a partitioned datasetARROW-7097 - [Rust][CI] Builds failing due to rust nightly formattingARROW-7100 - [C++] libjvm.so not found on ubuntu 19.04 with openjdk-11ARROW-7105 - [CI][Crossbow] Nightly homebrew-cpp job failsARROW-7106 - [Java] Fix the problem that flight perf test hangs endlesslyARROW-7117 - [C++][CI] Fix the hanging C++ tests in Windows 2019ARROW-7128 - [CI] Fedora cron jobs are failing because of wrong fedora versionARROW-7133 - [CI] Allow GH Actions to run on all branchesARROW-7142 - [C++] Compile error with GCC 5.4.0ARROW-7152 - [Java] Delete useless class DiffFunctionARROW-7157 - [R] Add validation, helpful error message to Object$new()ARROW-7158 - [C++][Visual Studio]Build config Error on non English Version visual studio.ARROW-7163 - [Doc] Fix double-and typosARROW-7164 - [CI] Dev cron github action is failing every 15 minutesARROW-7167 - [CI][Python] Add nightly tests for older pandas versions to Github ActionsARROW-7168 - [Python] pa.array() doesn't respect specified dictionary typeARROW-7170 - [C++] Bundled ORC fails linkingARROW-7180 - [CI] Java builds are not triggered on the master branchARROW-7181 - [Python][Nightly] Wheel builds could NOT find ArrowPythonARROW-7183 - [CI][Crossbow] Re-skip r-sanitizer nightly testsARROW-7187 - [C++][Doc] doxygen broken on master because of @ARROW-7188 - [C++][Doc] doxygen broken on master: missing param implicit_castsARROW-7189 - [CI][Crossbow] Nightly conda osx builds failARROW-7194 - [Rust] CSV Writer causing recursion errorsARROW-7199 - [Java] ConcurrentModificationException in BaseAllocator::getChildAllocatorsARROW-7200 - [C++][Flight] Running Arrow Flight benchmark on two hosts doesn't workARROW-7209 - [Python] tests with pandas master are failing now __from_arrow__ support landed in pandasARROW-7212 - "go test -bench=8192 -run=. ./math" failsARROW-7214 - [Python] unpickling a pyarrow table with dictionary fields crashesARROW-7217 - ARROW-7217: [CI][Python] Use correct python version in Github ActionsARROW-7225 - [C++] `*std::move(Result)` calls T copy constructor
ARROW-7249 - [CI] Release test fails in master due to new arrow-flight Rust crateARROW-7250 - [C++] Undefined symbols for StringToFloatConverter::Impl with clang 4.xARROW-7253 - [CI] Fix master failure with release testARROW-7254 - BaseVariableWidthVector#setSafe appears to make value offsets inconsistentARROW-7264 - [Java] RangeEqualsVisitor type check is not correctARROW-7266 - [Python] dictionary_encode() of a slice gives wrong resultARROW-7271 - [C++][Flight] Use the single parameter version of SetTotalBytesLimitARROW-7281 - [C++] AdaptiveIntBuilder::length() does not consider pending_pos_.ARROW-7282 - [Python] IO functions should raise FileNotFoundError when appropriateARROW-7291 - [Dev] Fix FORMAT_DIR in update-flatbuffers.shARROW-7294 - [Python] converted_type_name_from_enum(): Incorrect name for INT_64ARROW-7295 - [R] Fix bad test that causes failure on R < 3.5ARROW-7298 - [C++] cpp/thirdparty/download-dependencies.sh is brokenARROW-7314 - [Python] Compiler warning in pyarrowARROW-7318 - [C#] TimestampArray serialization failureARROW-7320 - [C++] Target arrow-type-benchmark failed to be built on bullx LinuxARROW-7327 - [CI] Failing C GLib and R buildbot buildersARROW-7328 - [CI] GitHub Actions should trigger on changes to GitHub Actions configurationARROW-7341 - [CI] Unbreak nightly Conda R jobARROW-7343 - [Java] Memory leak in Flight DoGet when client cancelsARROW-7349 - [C++] Fix the bug of parsing string hex valuesARROW-7353 - [C++] Disable -Wmissing-braces when building with clangARROW-7354 - [C++] TestHadoopFileSystem::ThreadSafety fails with sigabortARROW-7355 - [CI] Environment variables are defined twice for the fuzzit buildsARROW-7358 - [CI] [Dev] [C++] ccache disabled on conda-python-hdfsARROW-7359 - [C++][Gandiva] Don't throw error for locate function with start position exceeding string length, return 0 insteadARROW-7360 - [R] Can't use dplyr filter() with variables defined in parent scopeARROW-7361 - [Rust] Build directory is not passed to ci/scripts/rust_test.shARROW-7362 - [Python] ListArray.flatten() should take care of slicing offsetsARROW-7374 - [Dev] [C++] cuda-cpp docker image fails compiling ArrowARROW-7381 - [C++][Packaging] Iterator change broke manylinux1 wheelsARROW-7386 - [C#] Array offset does not work properlyARROW-7388 - [Python] Skip HDFS tests if libhdfs cannot be locatedARROW-7389 - [Python][Packaging] Remove pyarrow.s3fs import check from the recipeARROW-7393 - [Plasma] Fix plasma executable name in build for JavaARROW-7395 - [C++] Logical "or" with constants is a Clang warningARROW-7397 - [C++] Json white space length detection errorARROW-7404 - [C++][Gandiva] Fix utf8 char length error on Arm64ARROW-7406 - [Java] NonNullableStructVector#hashCode should pass hasher to child vectorsARROW-7407 - [Python] Failed to install pyarrow 0.15.1 on Python 3.8ARROW-7408 - [C++] Reference benchmarks fail compilingARROW-7435 - Security issue: ValidateOffsets() does not prevent buffer over-readARROW-7436 - [Archery] Fix benchmark default configurationARROW-7437 - [Java] ReadChannel#readFully does not set writer index correctlyARROW-7442 - [Ruby] Specifying column type as time causes segmentation faultARROW-7447 - [Java] ComplexCopier does incorrect copy in some casesARROW-7450 - [CI][C++] test-ubuntu-18.04-cpp-static failing with linking error in arrow-io-hdfs-testARROW-7458 - [GLib] incorrect build dependency in MakefileARROW-7471 - [Python] Cython flake8 failuresARROW-7472 - [Java] Fix some incorrect behavior in UnionListWriterARROW-7478 - [Rust] [DataFusion] Group by expression ignored unless paired with aggregate expressionARROW-7492 - [CI][Crossbow] Nightly homebrew-cpp job fails on Python installationARROW-7497 - [Python] Test asserts: pandas.util.testing is deprecated, use pandas.testing insteadARROW-7500 - [C++][Dataset] regex_error in hive partition on centos7 and opensuse42ARROW-7503 - [Rust] Rust builds are failing on masterARROW-7506 - [Java] JMH benchmarks should be called from main methodsARROW-7508 - [C#] DateTime32 Reading is BrokenARROW-7510 - [C++] Array::null_count() is not thread-compatibleARROW-7516 - [C#] .NET Benchmarks are brokenARROW-7518 - [Python] Use PYARROW_WITH_HDFS when building wheels, conda packagesARROW-7527 - [Python] pandas/feather tests failing on pandas masterARROW-7528 - [Python] The pandas.datetime class (import of datetime.datetime) and pandas.np are deprecatedARROW-7535 - [C++] ASAN failure in validationARROW-7543 - [R] arrow::write_parquet() code examples do not workARROW-7545 - [C++] [Dataset] Scanning dataset with dictionary type hangsARROW-7551 - [FlightRPC][C++] Flight test on macOS fails due to Homebrew gRPCARROW-7552 - [C++] TestSlowInputStream is flakyARROW-7554 - [C++] Unknown CMake command "externalproject_add".ARROW-7559 - [Rust] Possibly incorrect index check assertion in StringArray and BinaryArrayARROW-7561 - [Doc][Python] fix conda environment commandARROW-7563 - [Rust] failed to select a version for `byteorder`ARROW-7582 - [Rust][Flight] Unable to compile arrow.flight.protocol.rsARROW-7583 - [C++][Flight] Auth handler tests fragile on WindowsARROW-7591 - [Python] DictionaryArray.to_numpy returns dict of parts instead of numpy arrayARROW-7592 - [C++] Fix crashes on corrupt IPC inputARROW-7593 - [CI][Python] Python datasets failing on master / not run on CIARROW-7595 - [R][CI] R appveyor job fails due to pacman compression changeARROW-7596 - [Python] Only apply zero-copy DataFrame block optimizations when split_blocks=TrueARROW-7599 - [Java] Fix build break due to change in RangeEqualsVisitorARROW-7603 - [CI][Crossbow] Nightly centos 8 job failsARROW-7611 - [Packaging][Python] Artifacts patterns for wheel are wrongARROW-7612 - [Packaging][Python] Artifact paths for Conda on WIndows are wrongARROW-7614 - [Python] Slow performance in test_parquet.py::test_set_data_page_sizeARROW-7618 - [C++] Fix crashes or undefined behaviour on corrupt IPC inputARROW-7620 - [Rust] Windows builds failing due to flatbuffer compile errorARROW-7621 - [Doc] Doc build failsARROW-7634 - [Python] Dataset tests failing on Windows to parse file pathARROW-7638 - [Python] Segfault when inspecting dataset.Source with invalid file/partitioningARROW-7639 - [R] Cannot convert Dictionary Array to R when values aren't stringsARROW-7640 - [C++][Dataset] segfault when reading compressed Parquet files if build didn't include support for codecARROW-7647 - [C++] JSON reader fails to read arrays with few valuesARROW-7650 - [C++] Dataset tests not built on WindowsARROW-7651 - [CI][Crossbow] Nightly macOS wheel builds failARROW-7652 - [Python][Dataset] Insert implicit cast in ScannerBuilder.filterARROW-7661 - [Python] Non-optimal CSV chunking when no newline at endARROW-7689 - [C++] Sporadic Flight test crash on macOSARROW-7690 - [R] Cannot write parquet to OutputStreamARROW-7693 - [CI] Fix test-conda-python-3.7-spark-master nightly errorsARROW-7709 - [Python] Conversion from Table Column to Pandas loses name for TimestampsARROW-7714 - [Release] Variable expansion is missingARROW-7718 - [Release] Fix auto-retry in the binary release scriptARROW-7723 - [Python] StructArray timestamp type with timezone to_pandas convert errorARROW-7727 - [Python] Unable to read a ParquetDataset when schema validation is on.ARROW-8135 - [Python] Problem importing PyArrow on a clusterARROW-8638 - Arrow Cython API Usage Gives an error when calling CTable API EndpointsPARQUET-1692 - [C++] LogicalType::FromThrift error on Centos 7 RPMPARQUET-1692 - [C++] LogicalType::FromThrift error on Centos 7 RPMPARQUET-1693 - [C++] Build examples don't account for CMAKE compression feature flagsPARQUET-1702 - [C++] Make BufferedRowGroupWriter compatible with parquet encryptionPARQUET-1706 - [C++] Wrong dictionary_page_offset when writing only data pages via BufferedPageWriterPARQUET-1707 - [C++] parquet-arrow-test fails with undefined behaviour sanitizerPARQUET-1709 - [C++] Avoid unnecessary temporary std::shared_ptr copiesPARQUET-1715 - [C++] Add the Parquet code samples to CI + Refactor Parquet Encryption SamplesPARQUET-1720 - [C++] Parquet JSONPrint not showing version correctlyPARQUET-1747 - [C++] Access to ColumnChunkMetaData fails when encryption is onPARQUET-1766 - [C++] parquet NaN/null double statistics can result in endless loopPARQUET-1772 - [C++] ParquetFileWriter: Data overwritten when output stream opened in append modeNew Features and Improvements
ARROW-412 - [Format] Handling of buffer padding in the IPC metadataARROW-501 - [C++] Implement concurrent / buffering InputStream for streaming data use casesARROW-772 - [C++] Implement take kernel functionsARROW-843 - [C++] Implement Schema unification, merging unequal but equivalent schemasARROW-976 - [C++][Python] Provide API for defining and reading Parquet datasets with more ad hoc partition schemesARROW-1036 - [C++] Define abstract API for filtering Arrow streams (e.g. predicate evaluation)ARROW-1119 - [Python/C++] Implement NativeFile interfaces for Amazon S3ARROW-1175 - [Java] Implement/test dictionary-encoded subfieldsARROW-1456 - [Python] Run s3fs unit tests in Travis CIARROW-1562 - [C++] Numeric kernel implementations for add (+)ARROW-1638 - [Java] IPC roundtrip for null typeARROW-1900 - [C++] Add kernel functions for determining value range (maximum and minimum) of integer arraysARROW-2428 - [Python] Add API to map Arrow types (including extension types) to pandas ExtensionArray instances for to_pandas conversionsARROW-2602 - [Packaging] Automate build of development docker containersARROW-2863 - [Python] Add context manager APIs to RecordBatch*Writer/Reader classesARROW-3085 - [Rust] Add an adapter for parquet.ARROW-3408 - [C++] Add option to CSV reader to dictionary encode individual columns or all string / binary columnsARROW-3444 - [Python] Table.nbytes attributeARROW-3706 - [Rust] Add record batch reader trait.ARROW-3789 - [Python] Enable calling object in Table.to_pandas to "self-destruct" for improved memory useARROW-3808 - [R] Implement [.arrow::ArrayARROW-3813 - [R] lower level construction of Dictionary ArraysARROW-4059 - [Rust] Parquet/Arrow IntegrationARROW-4091 - [C++] Curate default list of CSV null spellingsARROW-4208 - [CI/Python] Have automatized tests for S3ARROW-4219 - [Rust] [Parquet] Implement ArrowReaderARROW-4223 - [Python] Support scipy.sparse integrationARROW-4224 - [Python] Support integration with pydata/sparse libraryARROW-4225 - [Format][C++] Add CSC sparse matrix supportARROW-4722 - [C++] Implement Bitmap class to modularize handling of bitmapsARROW-4748 - [Rust] [DataFusion] GROUP BY performance could be optimizedARROW-4930 - [Python] Remove LIBDIR assumptions in Python buildARROW-5180 - [Rust] IPC SupportARROW-5181 - [Rust] Create Arrow File readerARROW-5182 - [Rust] Create Arrow File writerARROW-5227 - [Rust] [DataFusion] Re-implement query execution with an extensible physical query planARROW-5277 - [C#] MemoryAllocator.Allocate(length: 0) should not return nullARROW-5333 - [C++] Fit build option summary into narrower consoleARROW-5366 - [Rust] Implement Duration and Interval ArraysARROW-5400 - [Rust] Test/ensure that reader and writer support zero-length record batchesARROW-5445 - [Website] Remove language that encourages pinning a versionARROW-5454 - [C++] Implement Take on ChunkedArray for DataFrame useARROW-5502 - [R] file readers should mmapARROW-5508 - [C++] Create reusable Iterator interface
ARROW-5523 - [Python] [Packaging] Use HTTPS consistently for downloading dependenciesARROW-5712 - [C++][Parquet] Arrow time32/time64/timestamp ConvertedType not being restored properlyARROW-5767 - [Format] Permit dictionary replacements in IPC protocolARROW-5801 - [CI] Dockerize (add to docker-compose) all Travis CI Linux tasksARROW-5802 - [CI] Dockerize "lint" Travis CI jobARROW-5804 - [C++] Dockerize C++ CI job with conda-forge toolchain, code coverage from Travis CIARROW-5805 - [Python] Dockerize (add to docker-compose) Python Travis CI jobARROW-5806 - [CI] Dockerize (add to docker-compose) Integration tests Travis CI entryARROW-5807 - [JS] Dockerize NodeJS Travis CI entryARROW-5808 - [GLib][Ruby] Dockerize (add to docker-compose) current GLib + Ruby Travis CI entryARROW-5809 - [Rust] Dockerize (add to docker-compose) Rust Travis CI buildARROW-5810 - [Go] Dockerize Travis CI Go buildARROW-5831 - [Release] Migrate and improve binary release verification scriptARROW-5839 - [Python] Test manylinux2010 in CIARROW-5855 - [Python] Add support for Duration typeARROW-5859 - [Python] Support ExtentionType on conversion to numpy/pandasARROW-5971 - [Website] Blog post introducing Arrow FlightARROW-5994 - [CI] [Rust] Create nightly releases of the Rust implementationARROW-6003 - [C++] Better input validation and error messaging in CSV readerARROW-6074 - [FlightRPC] Implement middlewareARROW-6091 - [Rust] [DataFusion] Implement parallel execution for limitARROW-6109 - [Integration] Docker image for integration testing can't be built on windowsARROW-6112 - [Java] Update APIs to support 64-bit address spaceARROW-6184 - [Java] Provide hash table based dictionary encoderARROW-6251 - [Developer] Add PR merge tool to apache/arrow-siteARROW-6257 - [C++] Add fnmatch compatible globbing functionARROW-6274 - [Rust] [DataFusion] Add support for writing results to CSVARROW-6277 - [C++][Parquet] Support reading/writing other Parquet primitive types to DictionaryArrayARROW-6283 - [Rust] [DataFusion] Implement operator to write query results to partitioned CSVARROW-6285 - [GLib] Add support for LargeBinary and LargeString typesARROW-6286 - [GLib] Add support for LargeList typeARROW-6299 - [C++] Simplify FileFormat classes to singletonsARROW-6321 - [Python] Ability to create ExtensionBlock on conversion to pandasARROW-6340 - [R] Implements low-level bindings to Dataset classesARROW-6341 - [Python] Implement low-level bindings for DatasetARROW-6352 - [Java] Add implementation of DenseUnionVector.ARROW-6367 - [C++][Gandiva] Implement string reverseARROW-6378 - [C++][Dataset] Implement TreeDataSourceARROW-6386 - [C++][Documentation] Explicit documentation of null slot interpretationARROW-6394 - [Java] Support conversions between delta vector and partial sum vectorARROW-6396 - [C++] Add ResolveNullOptions to Logical kernelsARROW-6398 - [C++] Consolidate ScanOptions and ScanContextARROW-6405 - [Python] Add std::move wrapper for use in CythonARROW-6452 - [Java] Override ValueVector toString() methodARROW-6463 - [C++][Python] Rename arrow::fs::Selector to FileSelectorARROW-6466 - [Developer] Refactor integration/integration_test.py into a proper Python packageARROW-6468 - [C++] Remove unused hashing routinesARROW-6473 - [Format] Clarify dictionary encoding edge casesARROW-6503 - [C++] Add an argument of memory pool object to SparseTensorConverterARROW-6508 - [C++] Add Tensor and SparseTensor factory function with validationsARROW-6515 - [C++] Clean type_traits.h definitionsARROW-6578 - [C++] Casting int64 to string columnsARROW-6592 - [Java] Add support for skipping decoding of columns/field in Avro converterARROW-6594 - [Java] Support logical type encodings from AvroARROW-6598 - [Java] Sort the code for ApproxEqualsVisitorARROW-6608 - [C++] Make default for ARROW_HDFS to be OFFARROW-6610 - [C++] Add ARROW_FILESYSTEM=ON/OFF CMake configuration flagARROW-6611 - [C++] Make ARROW_JSON=OFF the defaultARROW-6612 - [C++] Add ARROW_CSV CMake build flagARROW-6619 - [Ruby] Add support for building Gandiva::Expression by Arrow::Schema#build_expressionARROW-6624 - [C++] Add SparseTensor.ToTensor() methodARROW-6625 - [Python] Allow concat_tables to null or default fill missing columnsARROW-6631 - [C++] Do not build with any compression library dependencies by defaultARROW-6632 - [C++] Do not build with ARROW_COMPUTE=on and ARROW_DATASET=on by defaultARROW-6633 - [C++] Do not require double-conversion for default buildARROW-6634 - [C++] Do not require flatbuffers or flatbuffers_ep to buildARROW-6634 - [C++] Do not require flatbuffers or flatbuffers_ep to buildARROW-6635 - [C++] Do not require glog for default buildARROW-6636 - [C++] Do not build C++ command line utilities by defaultARROW-6637 - [C++] Zero-dependency default core buildARROW-6637 - [C++] Zero-dependency default core buildARROW-6646 - [Go] Amend NullType IPC implementation to append no buffers in RecordBatch messageARROW-6650 - [Rust] [Integration] Create methods to test Arrow files against Integration JSONARROW-6656 - [Rust] [DataFusion] Implement MIN and MAX aggregate expressionsARROW-6657 - [Rust] [DataFusion] Implement COUNT aggregate expressionARROW-6658 - [Rust] [DataFusion] Implement AVG aggregate expressionARROW-6659 - [Rust] [DataFusion] Refactor of HashAggregateExec to support custom mergeARROW-6662 - [Java] Implement equals/approxEquals API for VectorSchemaRootARROW-6671 - [C++] Sparse tensor namingARROW-6672 - [Java] Extract a common interface for dictionary buildersARROW-6685 - [C++/Python] S3 FileStat object's base_path and type depends on trailing slashARROW-6686 - [CI] Pull and push docker images to speed up the nightly buildsARROW-6688 - [Packaging] Include s3 support in the conda packagesARROW-6690 - [Rust] [DataFusion] HashAggregate without GROUP BY should use SIMDARROW-6692 - [Rust] [DataFusion] Update examples to use physical query planARROW-6693 - [Rust] [DataFusion] Update unit tests to use physical query planARROW-6694 - [Rust] [DataFusion] Update integration tests to use physical planARROW-6695 - [Rust] [DataFusion] Remove execution of logical planARROW-6696 - [Rust] [DataFusion] Implement simple math operations in physical query planARROW-6700 - [Rust] [DataFusion] Use new parquet arrow readerARROW-6707 - [Java] Improve the performance of JDBC adapters by using nullable informationARROW-6710 - [Java] Add JDBC adapter test to cover cases which contains some null valuesARROW-6711 - [C++] Consolidate Filter and Expression classesARROW-6721 - [JAVA] Avro adapter benchmark only runs once in JMHARROW-6722 - [Java] Provide a uniform way to get vector nameARROW-6729 - [C++] StlStringBuffer constructor is not zero-copyARROW-6730 - [CI] Use GitHub Actions for "C++ with clang 7" docker imageARROW-6731 - [CI] [Rust] Set up Github Action to run Rust testsARROW-6732 - [Java] Implement quick sort in a non-recursive way to avoid stack overflowARROW-6741 - [Release] Update changelog.py to use APACHE_ prefixed JIRA_USERNAME and JIRA_PASSWORD environment variablesARROW-6742 - [C++] Remove usage of boost::filesystem::path from arrow/io/hdfs_internal.ccARROW-6743 - [C++] Completely remove usage of boost::filesystem (except in hdfs_internal)ARROW-6744 - [Rust] Export JsonEqual trait in the array moduleARROW-6754 - [C++] Merge arrow/allocator.h and arrow/stl.h, or rename allocator.hARROW-6758 - [Release] Install ephemeral node/npm/npx in release verification scriptARROW-6764 - [C++] Add readahead iteratorARROW-6767 - [JS] lazily bind batches in scan/scanReverseARROW-6768 - [C++][Dataset] Implement dataset::Scan to Table helper functionARROW-6769 - [C++][Dataset] End to End dataset integration test caseARROW-6770 - [CI][Travis] Download Minio quietlyARROW-6777 - [GLib][CI] Unpin gobject-introspection gemARROW-6778 - [C++] Support DurationType in Cast kernelARROW-6782 - [C++] Build minimal core Arrow libraries without any Boost headersARROW-6784 - [C++][R] Move filter and take code from Rcpp to C++ libraryARROW-6787 - [CI] Decommission "C++ with clang 7 and system packages" Travis CI jobARROW-6788 - [CI] Migrate Travis CI lint job to GitHub ActionsARROW-6789 - [Python] Automatically box bytes/buffer-like values yielded from `FlightServerBase.do_action` in Result valuesARROW-6790 - [Release] Automatically disable integration test cases in release verificationARROW-6793 - [R] Arrow C++ binary packaging for LinuxARROW-6797 - [Release] Use a separately cloned arrow-site repository in the website post release scriptARROW-6802 - [Packaging][deb][RPM] Update qemu-user-static package URLARROW-6803 - [Rust] [DataFusion] Aggregate queries are slower with new physical query planARROW-6804 - [CI] [Rust] Migrate Travis Rust job to Github ActionsARROW-6807 - [Java][FlightRPC] Expose gRPC serviceARROW-6810 - [Website] Add docs for R package 0.15 releaseARROW-6811 - [R] Assorted post-0.15 release cleanupsARROW-6814 - [C++] Resolve compiler warnings occurred on release buildARROW-6822 - [Website] merge_pr.py is publishedARROW-6824 - [Plasma] Support batched create and seal requests for small objectsARROW-6825 - [C++] Rework CSV reader IO around readahead iteratorARROW-6831 - [R] Update R macOS/Windows builds for change in cmake compression defaultsARROW-6832 - [R] Implement Codec::IsAvailableARROW-6833 - [R][CI] Add crossbow job for full R autobrew macOS buildARROW-6836 - [Format] add a custom_metadata:[KeyValue] field to the Footer table in File.fbsARROW-6843 - [Website] Disable deploy on pull requestARROW-6847 - [C++] Add a range_expression interface to Iterator<>ARROW-6850 - [Java] Jdbc converter support Null typeARROW-6852 - [C++] memory-benchmark build failed on Arm64ARROW-6853 - [Java] Support vector and dictionary encoder use different hasher for calculating hashCodeARROW-6855 - [C++][Python][Flight] Implement Flight middlewareARROW-6862 - [Developer] Check pull request titleARROW-6863 - [Java] Provide parallel searcherARROW-6865 - [Java] Improve the performance of comparing an ArrowBuf against a byte arrayARROW-6866 - [Java] Improve the performance of calculating hash code for struct vectorARROW-6879 - [Rust] Add explicit SIMD for sum kernelARROW-6880 - [Rust] Add explicit SIMD for min/max kernelARROW-6881 - [Rust] Remove "array_ops" in favor of the "compute" sub-moduleARROW-6884 - [Python][Flight] Make server-side RPC exceptions more friendly?ARROW-6887 - [Java] Create prose documentation for using ValueVectorsARROW-6888 - [Java] Support copy operation for vector value comparatorsARROW-6889 - [Java] ComplexCopier enable FixedSizeList type & fix RangeEualsVisitor StackOverFlowARROW-6891 - [Rust] [Parquet] Add Utf8 support to ArrowReaderARROW-6902 - [C++] Add String*/Binary* support for Compare kernelsARROW-6904 - [Python] Implement MapArray and MapTypeARROW-6907 - [C++][Plasma] Allow Plasma store to batch notifications to clientsARROW-6911 - [Java] Provide composite comparatorARROW-6912 - [Java] Extract a common base class for avro converter consumersARROW-6916 - [Developer] Alphabetize task names in nightly Crossbow reportARROW-6918 - [R] Make docker-compose setup fasterARROW-6919 - [Python] Expose more builders in CythonARROW-6920 - [Python] create manylinux wheels for python3.8ARROW-6926 - [Python] Support __sizeof__ protocol for Python objectsARROW-6927 - [C++] Add gRPC version checkARROW-6928 - [Rust] Add FixedSizeList typeARROW-6930 - [Java] Create utility class for populating vector values used for test purpose onlyARROW-6932 - [Java] incorrect log on known extension typeARROW-6933 - [Java] Suppor linear dictionary encoderARROW-6936 - [Python] Improve error message when object of wrong type is givenARROW-6942 - [Developer] Add support for Parquet in pull request check by GitHub ActionsARROW-6943 - [Website] Translate Apache Arrow Flight introduction to JapaneseARROW-6944 - [Rust] Add StringTypeARROW-6949 - [Java] Fix promotable write to handle nullvectorsARROW-6951 - [C++][Dataset] Ensure column projection is passed to ParquetDataFragmentARROW-6952 - [C++][Dataset] Ensure expression filter is passed ParquetDataFragmentARROW-6954 - [Python] [CI] Add Python 3.8 to CI matrixARROW-6960 - [R] Add support for more compression codecs in Windows buildARROW-6961 - [C++][Gandiva] Add lower_utf8 function in GandivaARROW-6963 - [Packaging][Wheel][OSX] Use crossbow's command to deploy artifacts from travis buildsARROW-6964 - [C++][Dataset] Expose a nested parallel option for Scanner::ToTableARROW-6965 - [C++][Dataset] Optionally expose partition keys as materialized columnsARROW-6967 - [C++] Add filter expressions for IN, IS_VALIDARROW-6969 - [C++][Dataset] ParquetScanTask eagerly load fileARROW-6970 - [Packaging][RPM] Add support for CentOS 8ARROW-6973 - [C++][ThreadPool] Use perfect forwarding in SubmitARROW-6975 - [C++] Put make_unique in its own headerARROW-6980 - [R] dplyr backend for RecordBatch/TableARROW-6984 - [C++] Update LZ4 to 1.9.2 for CVE-2019-17543ARROW-6986 - [R] Add basic Expression classARROW-6987 - [CI] Travis OSX failing to install sdk headersARROW-6991 - [Packaging][deb] Add support for Ubuntu 19.10ARROW-6994 - [C++] Research jemalloc memory page reclamation configuration on macOS when background_thread option is unavailableARROW-6997 - [Packaging] Add support for RHELARROW-7000 - [C++][Gandiva] Handle empty inputs in string lower, upper functionsARROW-7003 - [Format] [Rust] Generate flatbuffers files in build scriptARROW-7004 - [Plasma] Make it possible to bump up object in LRU cacheARROW-7006 - [Rust] Bump flatbuffers version to avoid vulnerabilityARROW-7007 - [C++] Enable mmap option for LocalFsARROW-7014 - [Developer] Write script to verify Linux wheels given local environment with conda or virtualenvARROW-7015 - [Developer] Write script to verify macOS wheels given local environment with conda or virtualenvARROW-7016 - [Developer][Python] Write script to verify Windows wheels given local environment with condaARROW-7019 - [Java] Improve the performance of loading validity buffersARROW-7026 - [Java] Remove assertions in MessageSerializer/vector/writer/readerARROW-7031 - [Python] Expose the offsets of a ListArray in pythonARROW-7031 - [Python] Expose the offsets of a ListArray in pythonARROW-7032 - [Release] Run the python unit tests in the release verification scriptARROW-7034 - [CI][Crossbow] Skip known nightly failuresARROW-7035 - [R] Default arguments are unclear in write_parquet docsARROW-7036 - [C++] Version up ORC to avoid compile errorsARROW-7037 - [C++ ] Compile error on the combination of protobuf >= 3.9 and clangARROW-7039 - [Python] Typecheck expects pandas to be installedARROW-7047 - [C++][Dataset] Filter expressions should not require exact type matchARROW-7052 - [C++] Datasets example fails to build with ARROW_SHARED=OFFARROW-7054 - [Docs] Add option to override displayed docs version with an environment variableARROW-7057 - [C++] Add API to parse URI query stringsARROW-7058 - [C++] FileSystemDataSourceDiscovery should apply partition schemes relative to the base_dir of its selectorARROW-7060 - [R] Post-0.15.1 cleanupARROW-7061 - [C++][Dataset] FileSystemDiscovery with ParquetFileFormat should ignore files that aren't ParquetARROW-7062 - [C++] Parquet file parse error messages should include the file nameARROW-7064 - [R] Implement null typeARROW-7066 - [Python] support returning ChunkedArray from __arrow_array__ ?ARROW-7067 - [CI] Disable code coverage on Travis-CIARROW-7069 - [C++][Dataset] Replace ConstantPartitionScheme with PrefixDictionaryPartitionSchemeARROW-7070 - [Packaging][deb] Update package names for 1.0.0ARROW-7072 - [Java] Support concating validity bits efficientlyARROW-7082 - [Packaging][deb] Add apache-arrow-archive-keyringARROW-7086 - [C++] Provide a wrapper for invoking factories to produce a ResultARROW-7092 - [R] Add vignette for dplyr and datasetsARROW-7093 - [R] Support creating ScalarExpressions for more data typesARROW-7094 - [C++] FileSystemDataSource should use an owning pointer for fs::FilesystemARROW-7095 - [R] Better handling of unsupported filter and mutate expressions in dplyr methodsARROW-7096 - [C++] Add options structs for concatenation-with-promotion and schema unificationARROW-7098 - [Java] Improve the performance of comparing two memory blocksARROW-7099 - [C++] Disambiguate function calls in csv parser testARROW-7101 - [CI] Refactor docker-compose setup and use it with GitHub ActionsARROW-7103 - [R] Various minor cleanupsARROW-7107 - [C++][MinGW] Enable Flight on AppVeyorARROW-7110 - [GLib] Add filter support for GArrowTable, GArrowChunkedArray, and GArrowRecordBatchARROW-7111 - [GLib] Add take support for GArrowTable, GArrowChunkedArray, and GArrowRecordBatchARROW-7113 - [Rust] Buffer should accept memory owned by othersARROW-7116 - [CI] Use the docker repository provided by apache organisationARROW-7120 - [C++][CI] Add .ccache to the docker-compose volume mountsARROW-7146 - [R][CI] Various fixes and speedups for the R docker-compose setupARROW-7147 - [C++][Dataset] Refactor dataset's API to use Result
ARROW-7148 - [C++][Dataset] API cleanupARROW-7149 - [C++] Remove experimental status on filesystem APIsARROW-7155 - [Java][CI] add maven wrapper to make setup process simpleARROW-7159 - [CI] Run HDFS tests as cron taskARROW-7160 - [C++] Update string_view backportARROW-7161 - [C++] Migrate filesystem layer from Status to ResultARROW-7162 - [C++] Cleanup warnings in cmake_modules/SetupCxxFlags.cmakeARROW-7166 - [Java] Remove redundant code for Jdbc adaptersARROW-7169 - [C++] Vendor uriparser libraryARROW-7171 - [Ruby] Pass Array for Arrow::Table#filter
ARROW-7172 - [C++][Dataset] Improve format of Expression::ToStringARROW-7176 - [C++] Fix arrow::ipc compiler warningARROW-7178 - [C++] Vendor forward compatible std::optionalARROW-7185 - [R][Dataset] Add bindings for IN, IS_VALID expressionsARROW-7186 - [R] Add inline comments to document the dplyr codeARROW-7192 - [Rust] Implement Flight crateARROW-7193 - [Rust] Create Arrow stream readerARROW-7195 - [Ruby] Improve #filter, #take, and #is_inARROW-7196 - [Ruby] Remove needless BinaryArrayBuilder#append_valuesARROW-7197 - [Ruby] Suppress keyword argument related warnings with Ruby 2.7ARROW-7204 - [C++][Dataset] In expression should not require exact type matchARROW-7206 - [Java] Avoid string concatenation when calling Preconditions#checkArgumentARROW-7207 - [Rust] Update Generated Flatbuffer FilesARROW-7210 - [C++] Scalar cast should support time-based typesARROW-7211 - [Rust] [Parquet] Support writing to byte buffersARROW-7216 - [Java] Improve the performance of setting/clearing individual bitsARROW-7219 - [CI][Python] Install pickle5 in the conda-python docker image for python version 3.6ARROW-7227 - [Python] Provide wrappers for ConcatenateWithPromotion()ARROW-7228 - [Python] Expose RecordBatch.FromStructArray in Python.ARROW-7235 - [C++] Add Result to APIs to arrow/io
ARROW-7236 - [C++] Add Result to APIs to arrow/csv
ARROW-7240 - [C++] Add Result to APIs to arrow/util
ARROW-7246 - [CI][Python] wheel can't be built by SSL_ST_INIT errorARROW-7247 - [CI][Python] wheel can't be built by wget and OpenSSL errorARROW-7248 - [Rust] Automatically Regenerate IPC messages from FlatbuffersARROW-7255 - [CI] Run source release test on pull requestARROW-7257 - [CI] Homebrew formula is failed by openssl formula name updateARROW-7258 - [CI] Fuzzit job is failed by nonexistent directoryARROW-7259 - [Java] Support subfield encoder use different hasherARROW-7260 - [CI] Ubuntu 14.04 test is failed by user defined literalARROW-7261 - [Python] Python support for fixed size list typeARROW-7262 - [C++][Gandiva] Implement replace function in GandivaARROW-7263 - [C++][Gandiva] Implement locate and position functionsARROW-7268 - [Rust] Propagate `custom_metadata` field from IPC messageARROW-7269 - [C++] Fix arrow::parquet compiler warningARROW-7270 - [Go] preserve CSV reading behaviour, improve memory usageARROW-7274 - [C++] Add Result APIs to Decimal class
ARROW-7275 - [Ruby] Add support for Arrow::ListDataType.new(data_type)ARROW-7276 - [Ruby] Add support for building Arrow::ListArray from [[...]]ARROW-7277 - [Document] Add discussion about vector lifecycleARROW-7279 - [C++] Rename UnionArray::type_ids to UnionArray::type_codesARROW-7284 - [Java] ensure java implementation meets clarified dictionary specARROW-7289 - [C#] ListType constructor argument is redundantARROW-7290 - [C#] Implement ListArray BuilderARROW-7292 - [C++] [CI] [Dev] Add ASAN / UBSAN CI runARROW-7293 - [Dev] [C++] Persist ccache in docker-compose build volumesARROW-7296 - [Python] Add ORC api documentationARROW-7299 - [GLib] Use Result instead of StatusARROW-7303 - [C++] Refactor benchmarks to use new Result APIsARROW-7306 - [C++] Add Result-returning version of FileSystemFromUriARROW-7307 - [CI][GLib] Documentation isn't generatedARROW-7309 - [Python] Support HDFS federation viewfs://ARROW-7310 - [Python] Expose HDFS implementation for pyarrow.fsARROW-7311 - [Python] Return filesystem and path from URIARROW-7312 - [Rust] ArrowError should implement std::error:ErrorARROW-7317 - [C++] Migrate Iterator API to Result
ARROW-7319 - [C++] Refactor Iterator to yield Result
ARROW-7321 - [CI][GLib] Failed to build with GLib warningARROW-7322 - [CI][Python] Fall back to arrowdev dockerhub organization for manylinux imagesARROW-7323 - [CI][Rust] Nightly CI is failed by different toolchainARROW-7324 - [Rust] Add Timezone to TimestampARROW-7325 - [Rust] [Parquet] Update to parquet-format 2.6 and thrift 0.12ARROW-7329 - [Java] AllocationManager: Allow managing different types of memory other than those are allocated using NettyARROW-7333 - [CI][Rust] Remove duplicated nightly jobARROW-7334 - [CI][Python] macOS uses Python 2ARROW-7339 - [CMake] Thrift version not respected in CMake configuration version.txtARROW-7340 - [CI] Prune defunct appveyor build setupARROW-7344 - [Packaging][Python] Build manylinux2014 wheelsARROW-7346 - [CI] Explicit usage of ccache across the buildsARROW-7347 - [C++] Update bundled Boost to 1.71.0ARROW-7348 - [Rust] Add api to return references of buffer of null bitmap.ARROW-7351 - [Developer] Only suggest cpp-* fix versions when merging Parquet patchesARROW-7357 - [Go] migrate from pkg/errors to x/xerrorsARROW-7366 - [C++][Dataset] Use PartitionSchemeDiscovery in DataSourceDiscoveryARROW-7367 - [Python] Use np.full instead of np.array.repeat in ParquetDatasetPieceARROW-7368 - [Ruby] Use :arrow_file and :arrow_streaming for format nameARROW-7369 - [GLib] Add garrow_table_combine_chunksARROW-7370 - [C++] Old Protobuf with AUTO detection is failedARROW-7377 - [C++][Dataset] Simplify parquet column projectionARROW-7378 - [C++][Gandiva] Loop vectorization broken in IR optimizationARROW-7379 - [C++] Introduce SchemaBuilder companion class and Field::IsCompatibleWithARROW-7380 - [C++][Dataset] Implement DatasetFactoryARROW-7382 - [C++][Dataset] Refactor FsDsDiscovery constructorsARROW-7387 - [C#] Support ListType SerializationARROW-7392 - [Packaging] Add conda packaging tasks for python 3.8ARROW-7398 - [Packaging][Python] Conda builds are failing on macOSARROW-7399 - [C++][Gandiva] Gandiva does not pick runtime cpu featuresARROW-7402 - [C++] Add more information on CUDA errorARROW-7403 - [C++][JSON] Enable Rapidjson on Arm64 NeonARROW-7410 - [Python] [Doc] Document filesystem APIsARROW-7411 - [C++][Flight] Incorrect Arrow Flight benchmark outputARROW-7413 - [Python][Dataset] Add tests for PartitionSchemeDiscoveryARROW-7414 - [R][Dataset] Implement PartitionSchemeDiscoveryARROW-7415 - [C++][Dataset] Implement IpcFormat for sources composed of ipc filesARROW-7416 - [R][Nightly] Fix macos-r-autobrew build on R 3.6.2ARROW-7417 - [C++] Add a docker-compose entry for CUDA 10.1ARROW-7418 - [C++] Can't build with g++ 5.4.0 on Ubuntu 16.04ARROW-7420 - [C++] Migrate tensor related APIs to Result-returning versionARROW-7429 - [Java] Enhance code style checking for Java code (remove consecutive spaces)ARROW-7430 - [Python] Add more docstrings to dataset bindingsARROW-7431 - [Python] Add dataset API to reference docsARROW-7432 - [Python] Add higher-level datasets functionsARROW-7439 - [C++][Dataset] Remove dataset pointer aliasesARROW-7449 - [GLib] Make GObject Introspection optionalARROW-7452 - [GLib] Make GArrowTimeDataType abstractARROW-7453 - [Ruby] Add support for Arrow::NullArray#[]ARROW-7454 - [Ruby] Add support for saving/loading TSVARROW-7455 - [Ruby] Use Arrow::DataType.resolve for all GArrowDataType inputARROW-7456 - [C++] Add support for YYYY-MM-DDThh and YYYY-MM-DDThh:mm timestamp formatsARROW-7457 - [Doc] Fix typosARROW-7459 - [Python] Documentation lint is failedARROW-7460 - [Rust] Improve some kernels with autovectorisationARROW-7461 - [Java] Fix typos and spellingARROW-7463 - [Doc] Fix a broken link and typosARROW-7464 - [C++] Refine CpuInfo singleton with std::call_onceARROW-7465 - [C++] Add Arrow memory benchmark for Arm64ARROW-7468 - [Python] Fix typosARROW-7469 - [C++] Improve division related bit operationsARROW-7470 - [JS] Fix typosARROW-7474 - [Ruby] Save CSV files fasterARROW-7475 - [Rust] Create Arrow Stream writerARROW-7477 - [FlightRPC][Java] Flight gRPC service is missing reflection infoARROW-7479 - [Rust][Ruby][R] Fix typosARROW-7481 - [C#] Fix typosARROW-7482 - [C++] Fix typosARROW-7484 - [C++][Gandiva] Fix typosARROW-7485 - [C++][Plasma] Fix typosARROW-7487 - [Developer] Fix typosARROW-7488 - [GLib] Fix typos and broken linksARROW-7489 - [CI] Fix typosARROW-7490 - [Java] Avro converter should convert attributes and props to FieldType metadataARROW-7493 - [Python] Expose sum kernel in pyarrow.compute and support ChunkedArray inputsARROW-7498 - [C++][Dataset] Rename DataFragment/DataSource/PartitionSchemeARROW-7502 - [Integration] Remove Spark Integration patch that not needed anymoreARROW-7513 - [JS] Arrow Tutorial: Common data typesARROW-7514 - [C#] Make GetValueOffset ObsoleteARROW-7519 - [Python] Build wheels, conda packages with dataset supportARROW-7521 - [Rust] Remove tuple on FixedSizeList datatypeARROW-7523 - [Developer] Relax clang-tidy checkARROW-7526 - [C++][Compute]: Optimize small integer sortingARROW-7532 - [CI] Unskip brew test after Homebrew fixes it upstreamARROW-7537 - [CI][R] Nightly macOS autobrew job should be more verbose if it failsARROW-7538 - Clarify actual and desired size in AllocationManagerARROW-7540 - [C++] License files aren't installedARROW-7541 - [GLib] Install license filesARROW-7542 - [CI][C++] nproc isn't available on macOSARROW-7549 - [Java] Reorganize Flight modules to keep top level clean/organizedARROW-7550 - [R][CI] Run donttest examples in CIARROW-7557 - [C++][Compute] Validate sorting stability in random testARROW-7558 - [Packaging][deb][RPM] Use the host owner and group for artifactsARROW-7560 - [Rust] Reduce Rc/Refcell usageARROW-7565 - [Website] Add support for download URL redirectARROW-7566 - [CI] Use more recent Miniconda on AppVeyorARROW-7567 - [Java] Bump Checkstyle from 6.19 to 8.18ARROW-7567 - [Java] Bump Checkstyle from 6.19 to 8.18ARROW-7568 - [Java] Bump Apache Avro from 1.9.0 to 1.9.1ARROW-7569 - [Python] Add API to map Arrow types to pandas ExtensionDtypes for to_pandas conversionsARROW-7570 - [Java] Fix high severity issues reported by LGTMARROW-7571 - [Java] Correct minimal java version on READMEARROW-7572 - [Java] Enfore Maven 3.3+ as mentioned in READMEARROW-7573 - [Rust] Reduce boxing and cleanupARROW-7575 - [R] Linux binary packaging followupARROW-7576 - [C++][Dev] Improve fuzzing setupARROW-7577 - [C++][CI] Check fuzzer setup in CIARROW-7578 - [R] Add support for datasets with IPC files and with multiple sourcesARROW-7580 - [Website] 0.16 release postARROW-7581 - [R] Documentation/polishing for 0.16 releaseARROW-7590 - [C++] Managed files in thirdparty/ are ignoredARROW-7597 - [C++] Improvements to CMake configuration console summaryARROW-7600 - [C++][Parquet] Add a basic disabled unit test to excercise nesting functionalityARROW-7601 - [Doc] [C++] Update fuzzing documentationARROW-7602 - [Archery] Add more build optionsARROW-7613 - [Rust] Remove redundant `::` prefixesARROW-7622 - [Format] Mark Tensor and SparseTensor fields requiredARROW-7623 - [C++] Update generated flatbuffers filesARROW-7626 - [Parquet][GLib] Add support for version macrosARROW-7627 - [C++][Gandiva] Optimize string truncate functionARROW-7629 - [C++][CI] Add fuzz regression files to arrow-testingARROW-7630 - [C++][CI] Check fuzz crash regressions in CIARROW-7632 - [C++] [CI] Improve fuzzing seed corpusARROW-7635 - [C++] Add pkg-config support for each componentsARROW-7636 - [Python] Clean-up the pyarrow.dataset.partitioning() APIARROW-7644 - Add vcpkg installation instructionsARROW-7645 - [Packaging][deb][RPM] arm64 build by crossbow is brokenARROW-7648 - [C++] Sanitize local paths on WindowsARROW-7658 - [R] Support dplyr filtering on date/timeARROW-7659 - [Rust] Reduce Rc usageARROW-7660 - [C++][Gandiva] Optimise castVarchar(string, int) function for single byte charactersARROW-7665 - [R] linuxLibs.R should build in parallelARROW-7666 - [Packaging][deb] Always use NInja to reduce build timeARROW-7667 - [Packaging][deb] ubuntu-eoan is missing in nightly jobsARROW-7668 - [Packaging][RPM] Use NInja if possible to reduce build timeARROW-7670 - [Python][Dataset] Better ergonomics for the filter expressionsARROW-7671 - [Python][Dataset] Add bindings for the DatasetFactoryARROW-7674 - Add helpful message for captcha challenge in merge_arrow_pr.pyARROW-7682 - [Packaging][APT][Yum] Add support for arm64 APT/Yum repositoriesARROW-7683 - [Packaging] Set 0.16.0 as the next versionARROW-7686 - [Packaging][deb][RPM] Include more arrow-*.pcARROW-7687 - [C++] C++ developer document links in README are brokenARROW-7692 - [Rust] Several pattern matches are hard to readARROW-7694 - [Packaging][deb][RPM] Can't build repository packages for RCARROW-7695 - [Release] Update java versions to 0.16-SNAPSHOTARROW-7696 - [Release] Unit test on release branch is failedARROW-7697 - [Release] Add a test for updating Linux packages by 00-prepare.shARROW-7710 - [Release][C#] .NET download URL is redirectedARROW-7711 - [C#] Date32 test depends on system timezoneARROW-7715 - [Release][APT] Ignore some arm64 verificationsARROW-7716 - [Packaging][APT] Use the "main" component for Ubuntu 19.10ARROW-7719 - [Python][Dataset] Table equality check occasionally failsARROW-7724 - [Release][Yum] Ignore some arm64 verificationsARROW-7743 - [Rust] [Parquet] Support reading timestamp microsARROW-7768 - [Rust] Implement Length and TryClone traits for Cursor> in reader.rs
ARROW-8015 - [Python] Build 0.16.0 wheel install for Windows + Python 3.5 and publish to PyPIPARQUET-517 - [C++] Use arrow::MemoryPool for all heap allocationsPARQUET-1300 - [C++] Parquet modular encryptionPARQUET-1664 - [C++] Provide API to return metadata string from FileMetadata.PARQUET-1678 - [C++] Provide classes for reading/writing using input/output operatorsPARQUET-1688 - [C++] StreamWriter/StreamReader can't be built with g++ 4.8.5 on CentOS 7PARQUET-1689 - [C++] Stream API: Allow for columns/rows to be skipped when readingPARQUET-1701 - [C++] Stream API: Add support for optional fieldsPARQUET-1704 - [C++] Add re-usable encryption buffer to SerializedPageWriterPARQUET-1705 - [C++] Disable shrink-to-fit on the re-usable decryption bufferPARQUET-1712 - [C++] Stop using deprecated APIs in examplesPARQUET-1721 - [C++] Arrow dependency is missing in parquet.pcPARQUET-1734 - [C++] Fix typosPARQUET-1769 - [C++] Update to parquet-format 2.8.0Apache Arrow 0.15.1 (2019-11-01)
Bug Fixes
ARROW-6464 - [Java] Refactor FixedSizeListVector#splitAndTransfer with slice APIARROW-6728 - [C#] Support reading and writing Date32 and Date64 arraysARROW-6740 - [Python] Unable to delete closed MemoryMappedFile on WindowsARROW-6762 - [C++] JSON reader segfaults on newlineARROW-6795 - [C#] Reading large Arrow files in C# results in an exceptionARROW-6806 - [C++] Segfault deserializing ListArray containing null/empty listARROW-6809 - [RUBY] Gem does not install on macOS due to glib2 3.3.7 compilation failureARROW-6813 - [Ruby] Arrow::Table.load with headers=true leads to exception in Arrow 0.15ARROW-6834 - [C++] Pin gtest to 1.8.1 to triage failing Appveyor / MSVC buildARROW-6844 - [C++][Parquet][Python] List columns read broken with 0.15.0
ARROW-6857 - [Python][C++] Segfault for dictionary_encode on empty chunked_array (edge case)ARROW-6860 - [Python] Only link libarrow_flight.so to pyarrow._flightARROW-6861 - [Python] arrow-0.15.0 reading arrow-0.14.1-output Parquet dictionary column: Failure reading column: IOError: Arrow error: Invalid: Resize cannot downsizeARROW-6869 - [C++] Dictionary "delta" building logic in builder_dict.h produces invalid arraysARROW-6873 - [Python] Stale CColumn reference break Cython cimport pyarrowARROW-6874 - [Python] Memory leak in Table.to_pandas() when conversion to object dtypeARROW-6876 - [Python] Reading parquet file with many columns becomes slow for 0.15.0ARROW-6877 - [C++] Boost not found from the correct environmentARROW-6878 - [Python] pa.array() does not handle list of dicts with bytes keys correctly under python3ARROW-6882 - [Python] cannot create a chunked_array from dictionary_encoding resultARROW-6886 - [C++] arrow::io header nvcc compiler warningsARROW-6898 - [Java] Fix potential memory leak in ArrowWriter and several test classesARROW-6903 - [Python] Wheels broken after ARROW-6860 changesARROW-6905 - [Packaging][OSX] Nightly builds on MacOS are failing because of brew compile timeoutsARROW-6910 - [Python] pyarrow.parquet.read_table(...) takes up lots of memory which is not released until program exitsARROW-6922 - [Python] Pandas master build is failing (MultiIndex.levels change)ARROW-6937 - [Packaging][Python] Fix conda linux and OSX wheel nightly buildsARROW-6938 - [Python] Windows wheel depends on zstd.dll and libbz2.dll, which are not bundledARROW-6962 - [C++] [CI] Stop compiling with -WeverythingARROW-6977 - [C++] Only enable jemalloc background_thread if feature is supportedARROW-6983 - [C++] Threaded task group crashes sometimesARROW-7422 - [Python] Improper CPU flags failing pyarrow install in ARM devicesARROW-7423 - Pyarrow ARM install fails from source with no clear errorARROW-9349 - [Python] parquet.read_table causes crashes on Windows Server 2016 w/ Xeon ProcessorNew Features and Improvements
ARROW-6610 - [C++] Add ARROW_FILESYSTEM=ON/OFF CMake configuration flagARROW-6661 - [Java] Implement APIs like slice to enhance VectorSchemaRootARROW-6777 - [GLib][CI] Unpin gobject-introspection gemARROW-6852 - [C++] memory-benchmark build failed on Arm64ARROW-6927 - [C++] Add gRPC version checkARROW-6963 - [Packaging][Wheel][OSX] Use crossbow's command to deploy artifacts from travis builds
Apache Arrow 0.15.0 (2019-10-05)
New Features and Improvements
ARROW-453 - [C++] Add filesystem implementation for Amazon S3ARROW-517 - [C++] Verbose Array::EqualsARROW-750 - [Format] Add LargeBinary and LargeString typesARROW-1324 - [C++] Support ARROW_BOOST_VENDORED on Windows / MSVCARROW-1561 - [C++] Kernel implementations for "isin" (set containment)ARROW-1566 - [C++] Implement non-materializing sort kernelsARROW-1741 - [C++] Comparison function for DictionaryArray to determine if indices are "compatible"ARROW-1786 - [Format] List expected on-wire buffer layouts for each kind of Arrow physical type in specificationARROW-1789 - [Format] Consolidate specification documents and improve clarity for new implementation authorsARROW-1875 - [Java] Write 64-bit ints as strings in integration test JSON filesARROW-2006 - [C++] Add option to trim excess padding when writing IPC messagesARROW-2431 - [Rust] Schema fidelityARROW-2769 - [C++][Python] Deprecate and rename add_metadata methodsARROW-2931 - [Crossbow] Windows builds are attempting to run linux and osx packaging tasksARROW-3032 - [Python] Clean up NumPy-related C++ headersARROW-3204 - [R] Enable package to be made available on CRANARROW-3243 - [C++] Upgrade jemalloc to version 5ARROW-3246 - [Python][Parquet] direct reading/writing of pandas categoricals in parquetARROW-3325 - [Python] Support reading Parquet binary/string columns directly as DictionaryArrayARROW-3325 - [Python] Support reading Parquet binary/string columns directly as DictionaryArrayARROW-3531 - [Python] Deprecate Schema.field_by_name in favor of __getitem__ARROW-3538 - [Python] ability to override the automated assignment of uuid for filenames when writing datasetsARROW-3579 - [Crossbow] Unintuitive error message when remote branch has not been pushedARROW-3643 - [Rust] Optimize `push_slice` of `BufferBuilder`
ARROW-3710 - [Crossbow][Python] Run nightly tests against pandas masterARROW-3772 - [C++] Read Parquet dictionary encoded ColumnChunks directly into an Arrow DictionaryArrayARROW-3777 - [C++] Implement a mock "high latency" filesystemARROW-3817 - [R] $ method for RecordBatchARROW-3829 - [Python] Support protocols to extract Arrow objects from third-party classesARROW-3943 - [R] Write vignette for R packageARROW-4036 - [C++] Make status codes pluggableARROW-4095 - [C++] Implement optimizations for dictionary unification where dictionaries are prefixes of the unified dictionaryARROW-4111 - [Python] Create time types from Python sequences of integersARROW-4218 - [Rust] [Parquet] Implement ColumnReaderARROW-4220 - [Python] Add buffered input and output stream ASV benchmarks with simulated high latency IOARROW-4365 - [Rust] [Parquet] Implement RecordReaderARROW-4398 - [Python] Add benchmarks for Arrow<>Parquet BYTE_ARRAY serialization (read and write)ARROW-4473 - [Website] Add instructions to do a test-deploy of Arrow website and fix bugsARROW-4507 - [Format] Create outline and introduction for new document.ARROW-4508 - [Format] Copy content from Layout.rst to new document.ARROW-4509 - [Format] Copy content from Metadata.rst to new document.ARROW-4510 - [Format] copy content from IPC.rst to new document.ARROW-4511 - [Format] remove individual documents in favor of new document once all content is movedARROW-4648 - [C++/Question] Naming/organizational inconsistencies in cpp codebaseARROW-4648 - [C++/Question] Naming/organizational inconsistencies in cpp codebaseARROW-4649 - [C++/CI/R] Add (nightly) job that builds `brew install apache-arrow --HEAD`ARROW-4752 - [Rust] Add explicit SIMD vectorization for the divide kernelARROW-4810 - [Format][C++] Add "LargeList" type with 64-bit offsetsARROW-4841 - [C++] Persist CMake options in generated CMake configARROW-4860 - [C++] Build AWS C++ SDK for Windows in conda-forgeARROW-5134 - [R][CI] Run nightly tests against multiple R versionsARROW-5211 - [Format] Missing documentation under `Dictionary encoding` section on MetaData pageARROW-5216 - [CI] Add Appveyor badge to READMEARROW-5307 - [CI][GLib] Enable GTK-DocARROW-5337 - [C++] Add RecordBatch::field method, possibly deprecate "column"ARROW-5343 - [C++] Consider using Buffer for transpose maps in DictionaryType::Unify instead of std::vectorARROW-5344 - [C++] Use ArrayDataVisitor in implementation of dictionary unpacking in compute/kernels/cast.ccARROW-5351 - [Rust] Add support for take kernel functionsARROW-5358 - [Rust] Implement equality check for ArrayData and ArrayARROW-5380 - [C++] Fix and enable UBSan for unaligned accesses.ARROW-5439 - [Java] Utilize stream EOS in File formatARROW-5444 - [Release][Website] After 0.14 release, update what is an "official" releaseARROW-5458 - [C++] ARMv8 parallel CRC32c computation optimizationARROW-5480 - [Python] Pandas categorical type doesn't survive a round-trip through parquetARROW-5483 - [Java] add ValueVector constructors that take a Field objectARROW-5494 - [Python] Create FileSystem bindingsARROW-5505 - [R] Stop masking base R functions/rethink namespacingARROW-5527 - [C++] HashTable/MemoTable should use Buffer(s)/Builder(s) for heap dataARROW-5558 - [C++] Support Array::View on arrays with non-zero offsetsARROW-5559 - [C++] Introduce IpcOptions struct object for better API-stability when adding new optionsARROW-5564 - [C++] Add uriparser to conda-forgeARROW-5579 - [Java] shade flatbuffer dependencyARROW-5580 - [C++][Gandiva] Correct definitions of timestamp functions in GandivaARROW-5588 - [C++] Better support for building UnionArraysARROW-5594 - [C++] add support for UnionArrays to Take and FilterARROW-5610 - [Python] Define extension type API in Python to "receive" or "send" a foreign extension typeARROW-5646 - [Crossbow][Documentation] Move the user guide to the Sphinx documentationARROW-5681 - [FlightRPC] Wrap gRPC exceptions/statusesARROW-5686 - [R] Review R Windows CI buildARROW-5716 - [Developer] Improve merge PR script to acknowledge co-authorsARROW-5717 - [Python] Support dictionary unification when converting variable dictionaries to pandasARROW-5719 - [Java] Support in-place vector sortingARROW-5722 - [Rust] Implement std::fmt::Debug for ListArray, BinaryArray and StructArrayARROW-5734 - [Python] Dispatch to Table.from_arrays from pyarrow.table factory functionARROW-5736 - [Format][C++] Support small bit-width indices in sparse tensorARROW-5741 - [JS] Make numeric vector from functions consistent with TypedArray.fromARROW-5743 - [C++] Add CMake option to enable "large memory" unit testsARROW-5746 - [Website] Move website source out of apache/arrowARROW-5747 - [C++] Better column name and header support in CSV readerARROW-5758 - [C++][Gandiva] Support casting decimals to varchar and vice versaARROW-5762 - [Integration][JS] Integration Tests for Map TypeARROW-5777 - [C++] BasicDecimal128 is a small object it doesn't always make sense to pass by const refARROW-5778 - [Java] Extract the logic for vector data copying to the super classesARROW-5784 - [Release][GLib] Replace c_glib/ after running c_glib/autogen.sh in dev/release/02-source.shARROW-5786 - [Release] Use arrow-jni profile in dev/release/01-prepare.shARROW-5788 - [Rust] Use { version = "...", path = "../..." } for arrow and parquet dependenciesARROW-5789 - [C++] Small Warning/Linkage cleanupsARROW-5792 - [Rust] [Parquet] A visitor trait for parquet types.ARROW-5798 - [Packaging][deb] Update doc architectureARROW-5800 - [R] Dockerize R Travis CI tests so they can be run anywhere via docker-composeARROW-5803 - [C++] Dockerize C++ with clang 7 Travis CI unit test logicARROW-5812 - [Java] Refactor method name and param type in BaseIntVectorARROW-5813 - [C++] Support checking the equality of the different contiguous tensorsARROW-5814 - [Java] Implement a HashMap for DictionaryEncoder
ARROW-5827 - [C++] Require c-ares CMake configARROW-5828 - [C++] Add Protocol Buffers version checkARROW-5830 - [C++] Stop using memcmp in TensorEqualsARROW-5832 - [Java] Support search operations for vector dataARROW-5833 - [C++] Factor out status copying code from cast.ccARROW-5834 - [Java] Apply new hash map in DictionaryEncoderARROW-5835 - [Java] Support Dictionary Encoding for binary typeARROW-5841 - [Website] Add 0.14.0 release noteARROW-5842 - [Java] Revise the semantic of lastSet in ListVectorARROW-5843 - [Java] Improve the readability and performance of BitVectorHelper#getNullCountARROW-5844 - [Java] Support comparison & sort for more numeric typesARROW-5846 - [Java] Create Avro adapter module and add dependenciesARROW-5853 - [Python] Expose boolean filter kernel on ArrayARROW-5861 - [Java] Initial implement to convert Avro record with primitive typesARROW-5862 - [Java] Provide dictionary builderARROW-5864 - [Python] simplify cython wrapping of ResultARROW-5865 - [Release] Helper script for rebasing open pull requests on masterARROW-5866 - [C++] Remove duplicate library in cpp/BrewfileARROW-5867 - [C++][Gandiva] Add support for cast int to decimalARROW-5872 - Support mod(double, double) method in GandivaARROW-5876 - [FlightRPC] Implement basic auth across all languagesARROW-5877 - [FlightRPC] Fix auth incompatibilities between Python/JavaARROW-5880 - [C++] Update arrow parquet writer to use TypedBufferBuilderARROW-5881 - [Java] Provide functionalities to efficiently determine if a validity buffer has completely 1 bits/0 bitsARROW-5883 - [Java] Support dictionary encoding for List and Struct typeARROW-5888 - [Python][C++] Add metadata to store Arrow time zones in Parquet file metadataARROW-5891 - [C++][Gandiva] Remove duplicates in function registriesARROW-5892 - [C++][Gandiva] Support function aliasesARROW-5893 - [C++] Remove arrow::Column class from C++ libraryARROW-5897 - [Java] Remove duplicated logic in MapVectorARROW-5898 - [Java] Provide functionality to efficiently compute hash code for arbitrary memory segmentARROW-5900 - [Gandiva] [Java] Decimal precision,scale bounds checkARROW-5901 - [Rust] Implement PartialEq to compare array and json valuesARROW-5902 - [Java] Implement hash table and equals & hashCode API for dictionary encodingARROW-5903 - [Java] Set methods in DecimalVector are slowARROW-5904 - [Java] [Plasma] Fix compilation of Plasma Java clientARROW-5906 - [CI] Set -DARROW_VERBOSE_THIRDPARTY_BUILD=OFF in builds running in Travis CI, maybe all docker-compose builds by defaultARROW-5908 - [C#] ArrowStreamWriter doesn't align buffers to 8 bytesARROW-5909 - [Java] Optimize ByteFunctionHelpers equals & compare logicARROW-5911 - [Java] Make ListVector and MapVector create reader lazilyARROW-5917 - [Java] Redesign the dictionary encoderARROW-5918 - [Java] Add get to BaseIntVector interfaceARROW-5919 - [R] Add nightly tests for building r-arrow with dependencies from conda-forgeARROW-5920 - [Java] Support sort & compare for all variable width vectorsARROW-5924 - [C++][Plasma] It is not convenient to release a GPU objectARROW-5934 - [Python] Bundle arrow's LICENSE with the wheelsARROW-5937 - [Release] Stop parallel binary uploadARROW-5938 - [Release] Create branch for adding release note automaticallyARROW-5939 - [Release] Add support for generating vote email template separatelyARROW-5940 - [Release] Add support for re-uploading sign/checksum for binary artifactsARROW-5941 - [Release] Avoid re-uploading already uploaded binary artifactsARROW-5943 - [GLib][Gandiva] Add support for function aliasesARROW-5944 - [C++][Gandiva] Remove 'div' alias for 'divide'ARROW-5945 - [Rust] [DataFusion] Table trait should support building complete queriesARROW-5947 - [Rust] [DataFusion] Remove serde_json dependencyARROW-5948 - [Rust] [DataFusion] create_logical_plan should not call optimizerARROW-5955 - [Plasma] Support setting memory quotas per plasma client for better isolationARROW-5957 - [C++][Gandiva] Implement div function in GandivaARROW-5958 - [Python] Link zlib statically in the wheelsARROW-5961 - [R] Be able to run R-only tests even without C++ libraryARROW-5962 - [CI][Python] Do not test manylinux1 wheels in Travis CIARROW-5967 - [Java] DateUtility#timeZoneList is not correctARROW-5970 - [Java] Provide pointer to Arrow bufferARROW-5974 - [Python][C++] Enable CSV reader to read from concatenated gzip streamARROW-5975 - [C++][Gandiva] Add method to cast Date(in Milliseconds) to timestampARROW-5976 - [C++] RETURN_IF_ERROR(ctx) should be namespacedARROW-5977 - [C++] [Python] Method for read_csv to limit which columns are read?ARROW-5979 - [FlightRPC] Expose (de)serialization of protocol typesARROW-5985 - [Developer] Do not suggest setting Fix Version for point releases in dev/merge_arrow_pr.pyARROW-5986 - [Java] Code cleanup for dictionary encodingARROW-5988 - [Java] Avro adapter implement simple Record typeARROW-5997 - [Java] Support dictionary encoding for Union typeARROW-5998 - [Java] Open a document to track the API changesARROW-6000 - [Python] Expose LargeBinaryType and LargeStringTypeARROW-6008 - [Release] Don't parallelize the bintray upload scriptARROW-6009 - [Release][JS] Ignore NPM errors in the javascript release scriptARROW-6013 - [Java] Support range searcherARROW-6017 - [FlightRPC] Allow creating Locations with unknown schemesARROW-6020 - [Java] Refactor ByteFunctionHelper#hash with new added ArrowBufHasherARROW-6021 - [Java] Extract copyFrom and copyFromSafe methods to ValueVector interfaceARROW-6022 - [Java] Support equals API in ValueVector to compare two vectors equalARROW-6023 - [C++][Gandiva] Add functions in GandivaARROW-6024 - [Java] Provide more hash algorithmsARROW-6026 - [Doc] Add CONTRIBUTING.mdARROW-6030 - [Java] Efficiently compute hash code for ArrowBufPointerARROW-6031 - [Java] Support iterating a vector by ArrowBufPointerARROW-6034 - [C++][Gandiva] Add string functions in GandivaARROW-6035 - [Java] Avro adapter support convert nullable valueARROW-6036 - [GLib] Add support for skip rows and column_names CSV read optionARROW-6037 - [GLib] Add a missing version macroARROW-6039 - [GLib] Add garrow_array_filter()ARROW-6041 - [Website] Blog post announcing R package releaseARROW-6042 - [C++] Implement alternative DictionaryBuilder that always yields int32 indicesARROW-6045 - [C++] Benchmark for Parquet float and NaN encoding/decodingARROW-6048 - [C++] Add ChunkedArray::View which calls to Array::ViewARROW-6049 - [C++] Support using Array::View from compatible dictionary type to anotherARROW-6053 - [Python] RecordBatchStreamReader::Open2 cdef type signature doesn't match C++ARROW-6063 - [FlightRPC] Implement "half-closed" semantics for DoPutARROW-6065 - [C++] Reorganize parquet/arrow/reader.cc, remove code duplication, improve readabilityARROW-6069 - [Rust] [Parquet] Implement Converter to convert record reader to arrow primitive array.ARROW-6070 - [Java] Avoid creating new schema before IPC sendingARROW-6077 - [C++][Parquet] Build logical schema tree mapping Arrow fields to Parquet schema levelsARROW-6078 - [Java] Implement dictionary-encoded subfields for List typeARROW-6079 - [Java] Implement/test UnionFixedSizeListWriter for FixedSizeListVectorARROW-6080 - [Java] Support compare and search operation for BaseRepeatedValueVectorARROW-6083 - [Java] Refactor Jdbc adapter consume logicARROW-6084 - [Python] Support LargeListARROW-6085 - [Rust] [DataFusion] Create traits for phsyical query planARROW-6086 - [Rust] [DataFusion] Implement parallel execution for parquet scanARROW-6087 - [Rust] [DataFusion] Implement parallel execution for CSV scanARROW-6088 - [Rust] [DataFusion] Implement parallel execution for projectionARROW-6089 - [Rust] [DataFusion] Implement parallel execution for selectionARROW-6090 - [Rust] [DataFusion] Implement parallel execution for hash aggregateARROW-6093 - [Java] reduce branches in algo for first match in VectorRangeSearcherARROW-6094 - [Format][Flight] Add GetFlightSchema to Flight RPCARROW-6096 - [C++] Conditionally depend on boost regex libraryARROW-6097 - [Java] Avro adapter implement unions typeARROW-6100 - [Rust] Pin to specific Rust nightly releaseARROW-6101 - [Rust] [DataFusion] Create physical plan from logical planARROW-6102 - [Testing] Add partitioned CSV file to arrow-testing repoARROW-6104 - [Rust] [DataFusion] Don't allow bare_trait_objectsARROW-6105 - [C++][Parquet][Python] Add test case showing dictionary-encoded subfields in nested typeARROW-6113 - [Java] Support vector deduplicate functionARROW-6115 - [Python] support LargeList, LargeString, LargeBinary in conversion to pandasARROW-6118 - [Java] Replace google Preconditions with Arrow PreconditionsARROW-6121 - [Tools] Improve merge tool cli ergonomicARROW-6125 - [Python] Remove any APIs deprecated prior to 0.14.xARROW-6127 - [Website] Add favicons and meta tagsARROW-6128 - [C++] Can't build with g++ 8.3.0 by class-memaccess warningARROW-6130 - [Release] Use 0.15.0 as the next releaseARROW-6134 - [C++][Gandiva] Add concat function in GandivaARROW-6137 - [C++][Gandiva] Change output format of castVARCHAR(timestamp) in GandivaARROW-6137 - [C++][Gandiva] Change output format of castVARCHAR(timestamp) in GandivaARROW-6138 - [C++] Add a basic (single RecordBatch) implementation of DatasetARROW-6139 - [Documentation][R] Build R docs (pkgdown) site and add to arrow-siteARROW-6141 - [C++] Enable memory-mapping a file region that is offset from the beginning of the fileARROW-6142 - [R] Install instructions on linux could be clearerARROW-6143 - [Java] Unify the copyFrom and copyFromSafe methods for all vectorsARROW-6144 - [C++][Gandiva] Implement random function in GandivaARROW-6155 - [Java] Extract a super interface for vectors whose elements reside in continuous memory segmentsARROW-6156 - [Java] Support compare semantics for ArrowBufPointerARROW-6161 - [C++] Implements dataset::ParquetFile and associated Scan structuresARROW-6162 - [C++][Gandiva] Do not truncate string in castVARCHAR_varchar when out_len parameter is zeroARROW-6164 - [Docs][Format] Document project versioning schema and forward/backward compatibility policiesARROW-6172 - [Java] Provide benchmarks to set IntVector with different methodsARROW-6177 - [C++] Add Array::Validate()ARROW-6180 - [C++] Create InputStream that is an isolated reader of a segment of a RandomAccessFileARROW-6181 - [R] Only allow R package to install without libarrow on linuxARROW-6183 - [R] Document that you don't have to use tidyselect if you don't wantARROW-6185 - [Java] Provide hash table based dictionary builderARROW-6187 - [C++] fallback to storage type when writing ExtensionType to ParquetARROW-6188 - [GLib] Add garrow_array_is_in()ARROW-6192 - [GLib] Use the same SO version as C++ARROW-6194 - [Java] Add non-static approach in DictionaryEncoder making it easy to extend and reuseARROW-6196 - [Ruby] Add support for building Arrow::TimeNNArray by .newARROW-6197 - [GLib] Add garrow_decimal128_rescale()ARROW-6199 - [Java] Avro adapter avoid potential resource leak.ARROW-6203 - [GLib] Add garrow_array_sort_to_indices()ARROW-6204 - [GLib] Add garrow_array_is_in_chunked_array()ARROW-6206 - [Java][Docs] Document environment variables/java propertiesARROW-6209 - [Java] Extract set null method to the base class for fixed width vectorsARROW-6212 - [Java] Support vector rank operationARROW-6216 - [C++] Allow user to select the compression levelARROW-6217 - [Website] Remove needless _site/ directoryARROW-6219 - [Java] Add API for JDBC adapter that can convert less then the full result set at a time.ARROW-6220 - [Java] Add API to avro adapter to limit number of rows returned at a time.ARROW-6225 - [Website] Update arrow-site/README and any other places to point website contributors in right directionARROW-6229 - [C++] Add a DataSource implementation which scans a directoryARROW-6230 - [R] Reading in Parquet files are 20x slower than reading fst files in RARROW-6231 - [C++][Python] Consider assigning default column names when reading CSV file and header_rows=0ARROW-6232 - [C++] Rename Argsort kernel to SortToIndicesARROW-6237 - [R] Add option to set CXXFLAGS when compiling R package with $ARROW_R_CXXFLAGSARROW-6238 - [C++] Implement SimpleDataSource/SimpleDataFragmentARROW-6240 - [Ruby] Arrow::Decimal128Array returns BigDecimalARROW-6242 - [C++] Implements basic Dataset/Scanner/ScannerBuilderARROW-6243 - [C++] Implement basic Filter expression classesARROW-6244 - [C++] Implement Partition DataSourceARROW-6246 - [Website] Add link to R documentation siteARROW-6247 - [Java] Provide a common interface for float4 and float8 vectorsARROW-6249 - [Java] Remove useless class ByteArrayWrapperARROW-6250 - [Java] Implement ApproxEqualsVisitor comparing approx for floating pointARROW-6252 - [Python] Add pyarrow.Array.diff method that exposes arrow::DiffARROW-6253 - [Python] Expose "enable_buffered_stream" option from parquet::ReaderProperties in pyarrow.parquet.read_tableARROW-6258 - [R] Add macOS build scriptsARROW-6260 - [Website] Use deploy key on Travis to build and push to asf-siteARROW-6262 - [Developer] Show JIRA issue before mergingARROW-6264 - [Java] There is no need to consider byte order in ArrowBufHasherARROW-6265 - [Java] Avro adapter implement Array/Map/Fixed typeARROW-6267 - [Ruby] Add Arrow::Time for Arrow::Time{32,64}DataType valueARROW-6271 - [Rust] [DataFusion] Add example for running SQL against ParquetARROW-6272 - [Rust] [DataFusion] Add register_parquet convenience method to ExecutionContextARROW-6278 - [R] Read parquet files from raw vectorARROW-6279 - [Python] Add Table.slice method or allow slices in __getitem__ARROW-6284 - [C++] Allow references in std::tuple when converting tuple to arrow arrayARROW-6287 - [Rust] [DataFusion] Refactor TableProvider to return thread-safe BatchIteratorARROW-6288 - [Java] Implement TypeEqualsVisitor comparing vector type equals considering names and metadataARROW-6289 - [Java] Add empty() in UnionVector to create instanceARROW-6292 - [C++] Add an option to build with mimallocARROW-6294 - [C++] Use hyphen for plasma-store-server executableARROW-6295 - [Rust][DataFusion] ExecutionError Cannot compare Float32 with Float64ARROW-6296 - [Java] Cleanup JDBC interfaces and eliminate one memcopy for binary/varchar fieldsARROW-6297 - [Java] Compare ArrowBufPointers by unsinged integersARROW-6300 - [C++] Add io::OutputStream::Abort()ARROW-6303 - [Rust] Add a feature to disable SIMDARROW-6304 - [Java] Add description to each maven artifactARROW-6306 - [Java] Support stable sort by stable comparatorsARROW-6310 - [C++] Write 64-bit integers as strings in JSON integration test filesARROW-6311 - [Java] Make ApproxEqualsVisitor accept DiffFunction to make it more flexibleARROW-6313 - [Format] Tracking for ensuring flatbuffer serialized values are aligned in stream/files.ARROW-6314 - [C++] Implement changes to ensure flatbuffer alignment.ARROW-6314 - [C++] Implement changes to ensure flatbuffer alignment.ARROW-6315 - [Java] Make change to ensure flatbuffer reads are alignedARROW-6316 - [Go] Make change to ensure flatbuffer reads are alignedARROW-6317 - [JS] Implement changes to ensure flatbuffer alignmentARROW-6318 - [Integration] Update integration test to use generated binaries to ensure backwards compatibilityARROW-6319 - [C++] Extract the core of NumericTensor::Value as Tensor::Value
ARROW-6326 - [C++] Nullable fields when converting std::tuple to TableARROW-6328 - Click.option-s should have help textARROW-6329 - [Format] Add 4-byte "stream continuation" to IPC message format to align FlatbuffersARROW-6331 - [Java] Incorporate ErrorProne into the java buildARROW-6334 - [Java] Improve the dictionary builder API to return the position of the value in the dictionaryARROW-6335 - [Java] Improve the performance of DictionaryHashTableARROW-6336 - [Python] Clarify pyarrow.serialize/deserialize docstrings viz-a-viz relationship with Arrow IPC protocolARROW-6337 - [R] as_tibble in R API is a misnomerARROW-6338 - [R] Type function names don't match type namesARROW-6342 - [Python] Add pyarrow.record_batch factory function with same basic API / semantics as pyarrow.tableARROW-6346 - [GLib] Add garrow_array_view()ARROW-6347 - [GLib] Add garrow_array_diff_unified()ARROW-6350 - [Ruby] Remove Arrow::Struct and use Hash insteadARROW-6351 - [Ruby] Improve Arrow#values performanceARROW-6353 - [Python] Allow user to select compression level in pyarrow.parquet.write_tableARROW-6355 - [Java] Make range equal visitor reusableARROW-6356 - [Java] Avro adapter implement Enum type and nested Record typeARROW-6357 - [C++] S3: allow for background writesARROW-6358 - [C++] FileSystem::DeleteDir should make it optional to delete the directory itselfARROW-6360 - [R] Update support for compressionARROW-6362 - [C++] S3: more flexible credential optionsARROW-6365 - [R] Should be able to coerce numeric to integer with schemaARROW-6366 - [Java] Make field vectors final explicitlyARROW-6368 - [C++] Add RecordBatch projection functionalityARROW-6373 - [C++] Make FixedWidthBinaryBuilder consistent with other primitive fixed width buildersARROW-6375 - [C++] Extend ConversionTraits to allow efficiently appending list values in STL APIARROW-6379 - [C++] Do not append any buffers when serializing NullType for IPCARROW-6381 - [C++] BufferOutputStream::Write is slow for many small writesARROW-6383 - [Java] report outstanding child allocators on parent allocator closeARROW-6384 - [C++] Bump dependenciesARROW-6385 - [C++] Investigate xxh3ARROW-6391 - [Python][Flight] Add built-in methods on FlightServerBase to start server and wait for it to be availableARROW-6397 - [C++][CI] Fix S3 minio failureARROW-6401 - [Java] Implement dictionary-encoded subfields for Struct typeARROW-6402 - [C++] Suppress sign-compare warning with g++ 9.2.1ARROW-6403 - [Python] Expose FileReader::ReadRowGroups() to PythonARROW-6408 - [Rust] Use "if cfg!" pattern in SIMD kernel implementationsARROW-6413 - [R] Support autogenerating column namesARROW-6415 - [R] Remove usage of R CMD config CXXCPPARROW-6416 - [Python] Confusing API & documentation regarding chunksizesARROW-6417 - [C++][Parquet] Non-dictionary BinaryArray reads from Parquet format have slowed down since 0.11.xARROW-6419 - [Website] Blog post about Parquet dictionary performance work coming in 0.15.x releaseARROW-6422 - [Gandiva] Fix double-conversion linker issueARROW-6426 - [FlightRPC] Expose gRPC configuration knobs in FlightARROW-6427 - [GLib] Add support for column names autogeneration CSV read optionARROW-6438 - [R] Add bindings for filesystem APIARROW-6447 - [C++] Builds with ARROW_JEMALLOC=ON wait until jemalloc_ep is complete before building any libarrow .cc filesARROW-6450 - [C++] Use 2x reallocation strategy in arrow::BufferBuilder instead of 1.5xARROW-6451 - [Format] Add clarifications to Columnar.rst about the contents of "null" slots in Varbinary or List arraysARROW-6453 - [C++] More informative error messages from S3ARROW-6454 - [Developer] Add LLVM license to LICENSE.txt due to binary redistribution in packagesARROW-6458 - [Java] Remove value boxing/unboxing for ApproxEqualsVisitorARROW-6460 - [Java] Add benchmark and large fake data UT for avro adapterARROW-6462 - [C++] Can't build with bundled double-conversion on CentOS 6 x86_64ARROW-6465 - [Python] Improve Windows build instructionsARROW-6474 - [Python] Provide mechanism for python to write out old formatARROW-6475 - [C++] Don't try to dictionary encode dictionary arraysARROW-6477 - [Packaging][Crossbow] Use Azure Pipelines to build linux packagesARROW-6480 - [Developer] Add command to generate and send e-mail report for a Crossbow runARROW-6484 - [Java] Enable create indexType for DictionaryEncoding according to dictionary value countARROW-6487 - [Rust] [DataFusion] Create test utils moduleARROW-6489 - [Developer][Documentation] Fix merge script and readmeARROW-6490 - [Java] log error for leak in allocator closeARROW-6491 - [Java] fix master build failure caused by ErrorProneARROW-6494 - [C++][Dataset] Implement basic PartitionSchemeARROW-6504 - [Python][Packaging] Add mimalloc to conda packages for better performanceARROW-6505 - [Website] Add new committersARROW-6518 - [Packaging][Python] Flight failing in OSX Python wheel buildsARROW-6519 - [Java] Use IPC continuation token to mark EOSARROW-6524 - [Developer][Packaging] Nightly build report's subject should contain ArrowARROW-6525 - [C++] CloseFromDestructor() should perhaps not crashARROW-6526 - [C++] Poison data in PoolBuffer destructorARROW-6527 - [C++] Add OutputStream::Write() variant taking an owned bufferARROW-6531 - [Python] Add detach() method to buffered streamsARROW-6532 - [R] Write parquet files with compressionARROW-6533 - [R] Compression codec should take a "level"ARROW-6534 - [Java] Fix typos and spellingARROW-6539 - [R] Provide mechanism to write out old formatARROW-6540 - [R] Add Validate() methodsARROW-6541 - [Format][C++] Use two-part EOS and amend Format documentationARROW-6542 - [R] Add View() method to array typesARROW-6544 - [R] Documentation/polishing for 0.15 releaseARROW-6545 - [Go] Update Go IPC writer to use two-part EOS per mailing list discussionARROW-6546 - [C++] Add missing FlatBuffers source dependencyARROW-6549 - [C++] Switch back to latest jemalloc 5.xARROW-6556 - [Python] Prepare for pandas release without SparseDataFrameARROW-6556 - [Python] Prepare for pandas release without SparseDataFrameARROW-6557 - [Python] Always return pandas.Series from Array/ChunkedArray.to_pandas, propagate field names to Series from RecordBatch, TableARROW-6558 - [C++] Refactor Iterator to a type erased handleARROW-6559 - [Developer][C++] Add "archery" option to specify system toolchain for C++ buildsARROW-6563 - [Rust] [DataFusion] Create "merge" execution planARROW-6569 - [Website] Add support for auto deployment by GitHub ActionsARROW-6570 - [Python] Use MemoryPool to allocate memory for NumPy arrays in to_pandas callsARROW-6580 - [Java] Support comparison for unsigned integersARROW-6584 - [Python][Wheel] Bundle zlib again with the windows wheelsARROW-6588 - [C++] Suppress class-memaccess warning with g++ 9.2.1ARROW-6589 - [C++] Support BinaryType in MakeArrayOfNullARROW-6590 - [C++] Do not require ARROW_JSON=ON when ARROW_IPC=ONARROW-6591 - [R] Ignore .Rhistory files in source controlARROW-6599 - [Rust] [DataFusion] Implement SUM aggregate expressionARROW-6601 - [Java] Improve JDBC adapter performance & add benchmarkARROW-6605 - [C++] Add recursion depth control to fs::SelectorARROW-6606 - [C++] Construct tree structure from std::vector<fs::FileStats>ARROW-6609 - [C++] Add minimal build Dockerfile exampleARROW-6613 - [C++] Remove dependency on boost::filesystemARROW-6614 - [C++][Dataset] Implement FileSystemDataSourceDiscoveryARROW-6616 - [Website] Release announcement blog post for 0.15ARROW-6621 - [Rust][DataFusion] Examples for DataFusion are not executed in CIARROW-6629 - [Doc][C++] Document the FileSystem APIARROW-6630 - [Doc][C++] Document the file readers (CSV, JSON, Parquet, etc.)ARROW-6644 - [JS] Amend NullType IPC protocol to append no buffersARROW-6647 - [C++] Can't build with g++ 4.8.5 on CentOS 7 by member initializer for shared_ptrARROW-6648 - [Go] Expose the bitutil packageARROW-6649 - [R] print() methods for Table, RecordBatch, etc.ARROW-6653 - [Developer] Add support for auto JIRA link on pull requestARROW-6655 - [Python] Filesystem bindings for S3ARROW-6664 - [C++] Add option to build without SSE4.2ARROW-6665 - [Rust] [DataFusion] Implement numeric literal expressionsARROW-6667 - [Python] Avoid Reference Cycles in pyarrow.parquetARROW-6668 - [Rust] [DataFusion] Implement CAST expressionARROW-6669 - [Rust] [DataFusion] Implement physical expression for binary expressionsARROW-6675 - [JS] Add scanReverse function to dataFrame and filteredDataframeARROW-6683 - [Python] Add unit tests that validate cross-compatibility with pyarrow.parquet when fastparquet is installedARROW-6725 - [CI] Disable 3rdparty fuzzit nightly buildsARROW-6735 - [C++] Suppress sign-compare warning with g++ 9.2.1ARROW-6752 - [Go] implement Stringer for Null arrayARROW-6755 - [Release] Improvements to Windows release verification scriptARROW-6771 - [Packaging][Python] Missing pytest dependency from conda and wheel buildsPARQUET-1468 - [C++] Consolidate RecordReader, ColumnReader code pathsBug Fixes
ARROW-1184 - [Java] Dictionary.equals is not working correctlyARROW-2041 - [Python] pyarrow.serialize has high overhead for list of NumPy arraysARROW-2248 - [Python] Nightly or on-demand HDFS test buildsARROW-2317 - [Python] fix C linkage warningARROW-2490 - [C++] input stream locking inconsistentARROW-3176 - [Python] Overflow in Date32 column conversion to pandasARROW-3203 - [C++] Build error on Debian BusterARROW-3651 - [Python] Datetimes from non-DateTimeIndex cannot be deserializedARROW-3652 - [Python] CategoricalIndex is lost after reading backARROW-3762 - [C++] Parquet arrow::Table reads error when overflowing capacity of BinaryArrayARROW-3933 - [Python] Segfault reading Parquet files from GNOMADARROW-4187 - [C++] file-benchmark uses
ARROW-4746 - [C++/Python] PyDataTime_Date wrongly casted to PyDataTime_DateTimeARROW-4836 - [Python] "Cannot tell() a compressed stream" when using RecordBatchStreamWriterARROW-4848 - [C++] Static libparquet not compiled with -DARROW_STATIC on WindowsARROW-4880 - [Python] python/asv-build.sh is probably broken after CMake refactorARROW-4883 - [Python] read_csv() returns garbage if given file object in text modeARROW-5028 - [Python][C++] Creating list with pyarrow.array can overflow child builder
ARROW-5072 - [Python] write_table fails silently on S3 errorsARROW-5085 - [Python/C++] Conversion of dict encoded null column fails in parquet writing when using RowGroupsARROW-5086 - [Python] Space leak in ParquetFile.read_row_group()ARROW-5089 - [C++/Python] Writing dictionary encoded columns to parquet is extremely slow when using chunk sizeARROW-5103 - [Python] Segfault when using chunked_array.to_pandas on array different types (edge case)ARROW-5125 - [Python] Cannot roundtrip extreme dates through pyarrowARROW-5161 - [Python] Cannot convert struct type from Pandas object columnARROW-5220 - [Python] index / unknown columns in specified schema in Table.from_pandasARROW-5220 - [Python] index / unknown columns in specified schema in Table.from_pandasARROW-5292 - [C++] Static libraries are built on AppVeyorARROW-5300 - [C++] 0.13 FAILED to build with option -DARROW_NO_DEFAULT_MEMORY_POOLARROW-5374 - [Python] Misleading error message when calling pyarrow.read_record_batch on a complete IPC streamARROW-5414 - [C++] Using "Ninja" build system generator overrides default Release build type on WindowsARROW-5450 - [Python] TimestampArray.to_pylist() fails with OverflowError: Python int too large to convert to C longARROW-5471 - [C++][Gandiva]Array offset is ignored in Gandiva projectorARROW-5522 - [Packaging][Documentation] Comments out of date in python/manylinux1/build_arrow.shARROW-5525 - [C++][CI] Enable continuous fuzzingARROW-5560 - [C++][Plasma] Cannot create Plasma object after OutOfMemory errorARROW-5562 - [C++][Parquet] parquet writer does not handle negative zero correctlyARROW-5630 - [Python][Parquet] Table of nested arrays doesn't round tripARROW-5638 - [C++] cmake fails to generate Xcode project when Gandiva JNI bindings are enabledARROW-5651 - [Python] Incorrect conversion from strided Numpy array when other type is specifiedARROW-5682 - [Python] from_pandas conversion casts values to string inconsistentlyARROW-5731 - [CI] Turbodbc integration tests are failingARROW-5753 - [Rust] Fix test failure in CI code coverageARROW-5772 - [GLib][Plasma][CUDA] Plasma::Client#refer_object test is failedARROW-5775 - [C++] StructArray : cached boxed fields not thread-safeARROW-5776 - [Gandiva][Crossbow] Revert template to have commit ids.ARROW-5790 - [Python] Passing zero-dim numpy array to pa.array causes segfaultARROW-5817 - [Python] Use pytest marks for Flight test to avoid silently skipping unit tests due to import failuresARROW-5823 - [Rust] CI scripts miss --all-targets cargo argumentARROW-5824 - [Gandiva] [C++] Fix decimal nullARROW-5836 - [Java][OSX] Flight tests are failing: address already in useARROW-5838 - [C++][Flight][OSX] Building 3rdparty grpc cannot find OpenSSLARROW-5848 - [C++] SO versioning schema after release 1.0.0ARROW-5849 - [C++] Compiler warnings on mingw-w64ARROW-5850 - [CI][R] R appveyor job is broken after releaseARROW-5851 - [C++] Compilation of reference benchmarks failsARROW-5856 - [Python] linking 3rd party cython modules against pyarrow fails since 0.14.0ARROW-5860 - [Java] [Vector] Fix decimal byte setterARROW-5863 - [Python] Segmentation Fault via pytest-runnerARROW-5868 - [Python] manylinux2010 wheels have shared library dependency on liblz4ARROW-5870 - [C++] Development compile instructions need to include "make"ARROW-5873 - [Python] Segmentation fault when comparing schema with NoneARROW-5874 - [Python] pyarrow 0.14.0 macOS wheels depend on shared libs under /usr/local/optARROW-5878 - [Python][C++] Parquet reader not forward compatible for timestamps without timezoneARROW-5884 - [Java] Fix the get method of StructVectorARROW-5886 - [Python][Packaging] Manylinux1/2010 compliance issue with libzARROW-5887 - [C#] ArrowStreamWriter writes FieldNodes in wrong orderARROW-5889 - [Python][C++] Parquet backwards compat for timestamps without timezone brokenARROW-5894 - [C++] libgandiva.so.14 is exporting libstdc++ symbolsARROW-5899 - [Python][Packaging] Bundle uriparser.dll in windows wheelsARROW-5910 - [Python] read_tensor() fails on non-seekable streamsARROW-5921 - [C++][Fuzzing] Missing nullptr checks in IPCARROW-5923 - [C++] Fix int96 commentARROW-5925 - [Gandiva][C++] cast decimal to int should round upARROW-5930 - [FlightRPC] [Python] Flight CI tests are failingARROW-5930 - [FlightRPC] [Python] Flight CI tests are failingARROW-5935 - [C++] ArrayBuilders with mutable type are not robustly supportedARROW-5946 - [Rust] [DataFusion] Projection push down with aggregate producing incorrect resultsARROW-5952 - [Python] Segfault when reading empty table with category as pandas dataframeARROW-5959 - [C++][CI] Fuzzit does not know about branch + commit hashARROW-5960 - [C++] Boost dependencies are specified in wrong orderARROW-5963 - [R] R Appveyor job does not test changes in the C++ libraryARROW-5964 - [C++][Gandiva] Cast double to decimal with rounding returns 0ARROW-5965 - [Python] Regression: segfault when reading hive table with v0.14ARROW-5966 - [Python] Capacity error when converting large UTF32 numpy array to arrow arrayARROW-5968 - [Java] Remove duplicate Preconditions check in JDBC adapterARROW-5969 - [CI] [R] Lint failuresARROW-5973 - [Java] Variable width vectors' get methods should return null when the underlying data is nullARROW-5978 - [FlightRPC] [Java] Integration test client doesn't close buffersARROW-5989 - [C++][Python] pyarrow.lib.ArrowIOError: Unable to load libjvm when using openjdk-8ARROW-5990 - [Python] RowGroupMetaData.column misses bounds checkARROW-5992 - [C++] Array::View fails for string/utf8 as binaryARROW-5993 - [Python] Reading a dictionary column from Parquet results in disproportionate memory usageARROW-5996 - [Java] Avoid resource leak in flight serviceARROW-5999 - [C++] Required header files missing when built with -DARROW_DATASET=OFFARROW-6002 - [C++][Gandiva] TestCastFunctions does not test int64 casting`ARROW-6004 - [C++] CSV reader ignore_empty_lines option doesn't handle empty linesARROW-6005 - [C++] parquet::arrow::FileReader::GetRecordBatchReader() does not behave as documented since ARROW-1012ARROW-6006 - [C++] Empty IPC streams containing a dictionary are corruptARROW-6012 - [C++] Fall back on known Apache mirror for Thrift downloadsARROW-6015 - [Python] pyarrow wheel: `DLL load failed` when importing on windowsARROW-6016 - [Python] pyarrow get_library_dirs assertion errorARROW-6029 - [R] Improve R docs on how to fix library version mismatchARROW-6032 - [C++] CountSetBits doesn't ensure 64-bit aligned accessesARROW-6038 - [Python] pyarrow.Table.from_batches produces corrupted table if any of the batches were emptyARROW-6040 - [Java] Dictionary entries are required in IPC streams even when emptyARROW-6046 - [C++] Slice RecordBatch of String array with offset 0 returns whole batchARROW-6047 - [Rust] Rust nightly 1.38.0 builds failingARROW-6050 - [Java] Update out-of-date java/flight/README.mdARROW-6054 - pyarrow.serialize should respect the value of structured dtype of numpyARROW-6058 - [Python][Parquet] Failure when reading Parquet file from S3 with s3fsARROW-6059 - [Python] Regression memory issue when calling pandas.read_parquetARROW-6060 - [Python] too large memory cost using pyarrow.parquet.read_table with use_threads=TrueARROW-6061 - [C++] Cannot build libarrow without rapidjsonARROW-6066 - [Website] Fix blog post author headerARROW-6067 - [Python] Large memory test failuresARROW-6068 - [Python] Hypothesis test failure, Add StructType::Make that accepts vector of fieldsARROW-6073 - [C++] Decimal128Builder is not reset in Finish()ARROW-6082 - [Python] create pa.dictionary() type with non-integer indices type crashesARROW-6092 - [C++] Python 2.7: arrow_python_test failureARROW-6095 - [C++] Python subproject ignores ARROW_TEST_LINKAGEARROW-6108 - [C++] Appveyor Build_Debug configuration is hanging in C++ unit testsARROW-6116 - [C++][Gandiva] Fix bug in TimedTestFilterAdd2ARROW-6117 - [Java] Fix the set method of FixedSizeBinaryVectorARROW-6119 - [Python] PyArrow wheel import fails on Windows Python 3.7ARROW-6120 - [C++][Gandiva] including some headers causes decimal_test to failARROW-6126 - [C++] IPC stream reader handling of empty streams potentially not robustARROW-6132 - [Python] ListArray.from_arrays does not check validity of input arraysARROW-6135 - [C++] KeyValueMetadata::Equals should not be order-sensitiveARROW-6136 - [FlightRPC][Java] Don't double-close response streamARROW-6145 - [Java] UnionVector created by MinorType#getNewVector could not keep field type info properlyARROW-6148 - [C++][Packaging] Improve aarch64 supportARROW-6152 - [C++][Parquet] Write arrow::Array directly into parquet::TypedColumnWriter
ARROW-6153 - [R] Address parquet deprecation warningARROW-6158 - [Python] possible to create StructArray with type that conflicts with child array's typesARROW-6159 - [C++] PrettyPrint of arrow::Schema missing identation for first lineARROW-6160 - [Java] AbstractStructVector#getPrimitiveVectors fails to work with complex child vectorsARROW-6166 - [Go] Slice of slice causes index out of range panicARROW-6167 - [R] macOS binary R packages on CRAN don't have arrow_availableARROW-6168 - [C++] IWYU docker-compose job is brokenARROW-6170 - [R] "docker-compose build r" is slowARROW-6171 - [R] "docker-compose run r" failsARROW-6174 - [C++] Validate chunks in ChunkedArray::ValidateARROW-6175 - [Java] Fix MapVector#getMinorType and extend AbstractContainerVector addOrGet complex vector APIARROW-6178 - [Developer] Don't fail in merge script on bad primary author input in multi-author PRsARROW-6182 - [R] Add note to README about r-arrow conda installationARROW-6186 - [Packaging][C++] Plasma headers not included for ubuntu-xenial libplasma-dev debian packageARROW-6190 - [C++] Define and declare functions regardless of NDEBUGARROW-6193 - [GLib] Add missing require in testARROW-6200 - [Java] Method getBufferSizeFor in BaseRepeatedValueVector/ListVector not correctARROW-6202 - [Java] Exception in thread "main" org.apache.arrow.memory.OutOfMemoryException: Unable to allocate buffer of size 4 due to memory limit. Current allocation: 2147483646ARROW-6205 - [C++] ARROW_DEPRECATED warning when including io/interfaces.h from CUDA (.cu) sourceARROW-6208 - [Java] Correct byte order before comparing in ByteFunctionHelpersARROW-6210 - [Java] remove equals API from ValueVectorARROW-6211 - [Java] Remove dependency on RangeEqualsVisitor from ValueVector interfaceARROW-6214 - [R] Sanitizer errors triggered via R bindingsARROW-6215 - [Java] RangeEqualVisitor does not properly compare ZeroVectorARROW-6218 - [Java] Add UINT type test in integration to avoid potential overflowARROW-6223 - [C++] Configuration error with Anaconda Python 3.7.4ARROW-6224 - [Python] remaining usages of the 'data' attribute (from previous Column) cause warningsARROW-6227 - [Python] pyarrow.array() shouldn't coerce np.nan to stringARROW-6234 - [Java] ListVector hashCode() is not correctARROW-6241 - [Java] Failures on masterARROW-6255 - [Rust] [Parquet] Cannot use any published parquet crate due to parquet-format breaking changeARROW-6259 - [C++][CI] Flatbuffers-related failures in CI on macOSARROW-6263 - [Python] RecordBatch.from_arrays does not check array types against a passed schemaARROW-6266 - [Java] Resolve the ambiguous method overload in RangeEqualsVisitorARROW-6268 - Empty buffer should have a valid addressARROW-6269 - [C++][Fuzzing] IPC reads do not check decimal precisionARROW-6270 - [C++][Fuzzing] IPC reads do not check buffer indicesARROW-6290 - [Rust] [DataFusion] sql_csv example errors when runningARROW-6291 - [C++] CMake ignores ARROW_PARQUETARROW-6293 - [Rust] datafusion 0.15.0-SNAPSHOT errorARROW-6301 - [Python] atexit: pyarrow.lib.ArrowKeyError: 'No type extension with name arrow.py_extension_type found'ARROW-6302 - [Python][Parquet] Reading dictionary type with serialized Arrow schema does not restore "ordered" type propertyARROW-6309 - [C++] Parquet tests and executables are linked staticallyARROW-6323 - [R] Expand file paths when passing to readersARROW-6325 - [Python] wrong conversion of DataFrame with boolean valuesARROW-6330 - [C++] Include missing headers in api.hARROW-6332 - [Java][C++][Gandiva] Handle size of varchar vectors correctlyARROW-6339 - [Python][C++] Rowgroup statistics for pd.NaT array ill definedARROW-6343 - [Java] [Vector] Fix allocation helperARROW-6344 - [C++][Gandiva] substring does not handle multibyte charactersARROW-6345 - [C++][Python] "ordered" flag seemingly not taken into account when comparing DictionaryType values for equalityARROW-6348 - [R] arrow::read_csv_arrow namespace error when package not loadedARROW-6354 - [C++] Building without Parquet failsARROW-6363 - [R] segfault in Table__from_dots with unexpected schemaARROW-6364 - [R] Handling unexpected input to time64() et alARROW-6369 - [Python] Support list-of-boolean in Array.to_pandas conversionARROW-6371 - [Doc] Row to columnar conversion example mentions arrow::Column in commentsARROW-6372 - [Rust][Datafusion] Casting from Un-signed to Signed Integers not supportedARROW-6376 - [Developer] PR merge script has "master" target ref hard-codedARROW-6387 - [Archery] Errors with makeARROW-6392 - [Python][Flight] list_actions Server RPC is not tested in test_flight.py, nor is return value validatedARROW-6395 - [Python] Bug when using bool arrays with stride greater than 1ARROW-6406 - [C++] jemalloc_ep fails for offline buildARROW-6411 - [C++][Parquet] DictEncoderImpl::PutIndicesTyped has bad performance on some systems
ARROW-6412 - [C++] arrow-flight-test can crash because of port allocationARROW-6418 - [C++] Plasma cmake targets are not exportedARROW-6423 - [Python] pyarrow.CompressedOutputStream() never completes with compression='snappy'ARROW-6424 - [C++][Fuzzing] Fuzzit nightly is brokenARROW-6425 - [C++] ValidateArray fail for slice of list arrayARROW-6428 - [CI][Crossbow] Nightly turbodbc job failsARROW-6430 - [CI][Crossbow] Nightly R docker job failsARROW-6431 - [Python] Test suite fails without pandas installedARROW-6432 - [CI][Crossbow] Remove alpine crossbow jobsARROW-6433 - [CI][Crossbow] Nightly java docker job failsARROW-6434 - [CI][Crossbow] Nightly HDFS integration job failsARROW-6435 - [CI][Crossbow] Nightly dask integration job failsARROW-6440 - [CI][Crossbow] Nightly ubuntu, debian, and centos package builds failARROW-6441 - [CI][Crossbow] Nightly Centos 6 job failsARROW-6442 - [CI][Crossbow] Nightly gandiva jar osx build failsARROW-6443 - [CI][Crossbow] Nightly conda osx builds failARROW-6444 - [CI][Crossbow] Nightly conda Windows builds fail (time out)ARROW-6446 - [OSX][Python][Wheel] Turn off ORC feature in the wheel building scriptsARROW-6449 - [R] io "tell()" methods are inconsistently named and untestedARROW-6457 - [C++] CMake build locally fails with MSVC 2015 build generatorARROW-6461 - [Java] EchoServer can close socket before client has finished readingARROW-6472 - [Java] ValueVector#accept may has potential cast exceptionARROW-6476 - [Java][CI] Travis java all-jdks job is brokenARROW-6478 - [C++] Roll back to jemalloc stable-4 branch until performance issues in 5.2.x addressedARROW-6481 - [Python][C++] Bad performance of read_csv() with column_typesARROW-6488 - [Python] pyarrow.NULL equals to itselfARROW-6492 - [Python] file written with latest fastparquet cannot be read with latest pyarrowARROW-6502 - [GLib][CI] MinGW failure in CIARROW-6506 - [C++] Validation of ExtensionType with nested type failsARROW-6509 - [C++][Gandiva] Re-enable Gandiva JNI tests and fix Travis CI failureARROW-6509 - [C++][Gandiva] Re-enable Gandiva JNI tests and fix Travis CI failureARROW-6520 - [Python] Segmentation fault on writing tables with fixed size binary fieldsARROW-6522 - [Python] Test suite fails with pandas 0.23.4, pytest 3.8.1ARROW-6530 - [CI][Crossbow][R] Nightly R job doesn't install all dependenciesARROW-6550 - [C++] Filter expressions PR failing manylinux package buildsARROW-6551 - [Python] Dask Parquet integration test failureARROW-6552 - [C++] boost::optional in STL test fails compiling in gcc 4.8.2ARROW-6560 - [Python] Failures in *-nopandas integration testsARROW-6561 - [Python] pandas-master integration test failureARROW-6562 - [GLib] Fix wrong sliced data of GArrowBufferARROW-6564 - [Python] Do not require pandas for invoking Array.__array__ARROW-6565 - [Rust] [DataFusion] Intermittent test failure due to temp dir already existingARROW-6568 - [C++][Python][Parquet] pyarrow.parquet crash writing zero-chunk dictionary-type columnARROW-6572 - [C++] Reading some Parquet data can return uninitialized memoryARROW-6573 - [Python] Segfault when writing to parquetARROW-6576 - [R] Fix sparklyr integration testsARROW-6586 - [Python][Packaging] Windows wheel builds failing with "DLL load failure"ARROW-6597 - [Python] Segfault in test_pandas with Python 2.7ARROW-6618 - [Python] Reading a zero-size buffer can segfaultARROW-6620 - [Python][CI] pandas-master build failing due to removal of "to_sparse" methodARROW-6622 - [C++][R] SubTreeFileSystem path error on WindowsARROW-6623 - [CI][Python] Dask docker integration test broken perhaps by statistics-related changeARROW-6639 - [Packaging][RPM] Add support for CentOS 7 on aarch64ARROW-6640 - [C++] Error when BufferedInputStream Peek more than bytes bufferedARROW-6641 - [C++] Remove Deprecated WriteableFile warningARROW-6642 - [Python] chained access of ParquetDataset's metadata segfaultsARROW-6651 - [R] Fix R conda jobARROW-6652 - [Python] to_pandas conversion removes timezone from typeARROW-6652 - [Python] to_pandas conversion removes timezone from typeARROW-6660 - [Rust] [DataFusion] Minor docs update for 0.15.0 releaseARROW-6670 - [CI][R] Fix fix for R nightly jobsARROW-6674 - [Python] Fix or ignore the test warningsARROW-6677 - [FlightRPC][C++] Document using Flight in C++ARROW-6678 - [C++] Regression in Parquet file compatibility introduced by ARROW-3246ARROW-6679 - [RELEASE] autobrew license in LICENSE.txt is not acceptableARROW-6682 - [C#] Arrow R/C++ hangs reading binary file generated by C#ARROW-6687 - [Rust] [DataFusion] Query returns incorrect row countARROW-6687 - [Rust] [DataFusion] Query returns incorrect row countARROW-6701 - [C++][R] Lint failing on R cpp codeARROW-6703 - [Packaging][Linux] Restore ARROW_VERSION environment variableARROW-6705 - [Rust] [DataFusion] README has invalid github URLARROW-6709 - [JAVA] Jdbc adapter currentIndex should increment when value is nullARROW-6714 - [R] Fix untested RecordBatchWriter caseARROW-6716 - [CI] [Rust] New 1.40.0 nightly causing builds to failARROW-6748 - [RUBY] gem compilation errorARROW-6751 - [CI] ccache doesn't cache on Travis-CIARROW-6760 - [C++] JSON: improve error message when column changed typeARROW-6773 - [C++] Filter kernel returns invalid data when filtering with an Array sliceARROW-6796 - Certain moderately-sized (~100MB) default-Snappy-compressed Parquet files take enormous memory and long time to load by pyarrow.parquet.read_tableARROW-7112 - Wrong contents when initializinga pyarrow.Table from boolean DataFramePARQUET-1623 - [C++] Invalid memory access with a magic number of recordsPARQUET-1631 - [C++] ParquetInputWrapper::GetSize always returns 0PARQUET-1640 - [C++] parquet-encoding-benchmark crashesApache Arrow 0.14.1 (2019-07-22)
Bug Fixes
ARROW-5775 - [C++] StructArray : cached boxed fields not thread-safeARROW-5790 - [Python] Passing zero-dim numpy array to pa.array causes segfaultARROW-5791 - [Python] pyarrow.csv.read_csv hangs + eats all RAMARROW-5816 - [Release] Parallel curl does not work reliably in verify-release-candidate-shARROW-5836 - [Java][OSX] Flight tests are failing: address already in useARROW-5838 - [C++][Flight][OSX] Building 3rdparty grpc cannot find OpenSSLARROW-5849 - [C++] Compiler warnings on mingw-w64ARROW-5850 - [CI][R] R appveyor job is broken after releaseARROW-5851 - [C++] Compilation of reference benchmarks failsARROW-5856 - [Python] linking 3rd party cython modules against pyarrow fails since 0.14.0ARROW-5863 - [Python] Segmentation Fault via pytest-runnerARROW-5868 - [Python] manylinux2010 wheels have shared library dependency on liblz4ARROW-5873 - [Python] Segmentation fault when comparing schema with NoneARROW-5874 - [Python] pyarrow 0.14.0 macOS wheels depend on shared libs under /usr/local/optARROW-5878 - [Python][C++] Parquet reader not forward compatible for timestamps without timezoneARROW-5886 - [Python][Packaging] Manylinux1/2010 compliance issue with libzARROW-5887 - [C#] ArrowStreamWriter writes FieldNodes in wrong orderARROW-5889 - [Python][C++] Parquet backwards compat for timestamps without timezone brokenARROW-5899 - [Python][Packaging] Bundle uriparser.dll in windows wheelsARROW-5921 - [C++][Fuzzing] Missing nullptr checks in IPCPARQUET-1623 - [C++] Invalid memory access with a magic number of records
New Features and Improvements
ARROW-5101 - [Packaging] Avoid bundling static libraries in Windows conda packagesARROW-5380 - [C++] Fix and enable UBSan for unaligned accesses.ARROW-5564 - [C++] Add uriparser to conda-forgeARROW-5609 - [C++] Set CMP0068 CMake policy to avoid macOS warningsARROW-5784 - [Release][GLib] Replace c_glib/ after running c_glib/autogen.sh in dev/release/02-source.shARROW-5785 - [Rust] Rust datafusion implementation should not depend on rustylineARROW-5787 - [Release][Rust] Use local modules to verify RCARROW-5793 - [Release] Avoid duplicate known host SSH error in dev/release/03-binary.shARROW-5794 - [Release] Skip uploading already uploaded binariesARROW-5795 - [Release] Add missing waits on uploading binariesARROW-5796 - [Release][APT] Update expected package listARROW-5797 - [Release][APT] Update supported distributionsARROW-5820 - [Release] Remove undefined variable check from verify scriptARROW-5827 - [C++] Require c-ares CMake configARROW-5828 - [C++] Add Protocol Buffers version checkARROW-5866 - [C++] Remove duplicate library in cpp/BrewfileARROW-5877 - [FlightRPC] Fix auth incompatibilities between Python/JavaARROW-5904 - [Java] [Plasma] Fix compilation of Plasma Java clientARROW-5908 - [C#] ArrowStreamWriter doesn't align buffers to 8 bytesARROW-5934 - [Python] Bundle arrow's LICENSE with the wheelsARROW-5937 - [Release] Stop parallel binary uploadARROW-5938 - [Release] Create branch for adding release note automaticallyARROW-5939 - [Release] Add support for generating vote email template separatelyARROW-5940 - [Release] Add support for re-uploading sign/checksum for binary artifactsARROW-5941 - [Release] Avoid re-uploading already uploaded binary artifactsARROW-5958 - [Python] Link zlib statically in the wheels
Apache Arrow 0.14.0 (2019-07-04)
New Features and Improvements
ARROW-258 - [Format] clarify definition of Buffer in context of RPC, IPC, FileARROW-653 - [Python / C++] Add debugging function to print an array's buffer contents in hexadecimalARROW-767 - [C++] Adopt FileSystem abstractionARROW-835 - [Format] Add Timedelta type to describe time intervalsARROW-840 - [Python] Provide Python API for creating user-defined data types that can survive Arrow IPCARROW-973 - [Website] Add FAQ page about projectARROW-1012 - [C++] Create a configurable implementation of RecordBatchReader that reads from Apache Parquet filesARROW-1207 - [C++] Implement Map logical typeARROW-1261 - [Java] Add container type for Map logical typeARROW-1278 - Integration tests for Fixed Size List typeARROW-1279 - [Integration][Java] Integration tests for Map typeARROW-1280 - [C++] Implement Fixed Size List typeARROW-1349 - [Packaging] Provide APT and Yum repositoriesARROW-1496 - [JS] Upload coverage data to codecov.ioARROW-1558 - [C++] Implement boolean selection kernelsARROW-1587 - [Format] Add metadata for user-defined logical typesARROW-1774 - [C++] Add "view" function to create zero-copy views for compatible types, if supportedARROW-1833 - [Java] Add accessor methods for data buffers that skip null checkingARROW-1957 - [Python] Write nanosecond timestamps using new NANO LogicalType Parquet unitARROW-1983 - [Python] Add ability to write parquet `_metadata` fileARROW-2057 - [Python] Configure size of data pages in pyarrow.parquet.write_tableARROW-2102 - [C++] Implement take kernel functions - primitive value typeARROW-2103 - [C++] Implement take kernel functions - string/binary value typeARROW-2104 - [C++] Implement take kernel functions - nested array value typeARROW-2105 - [C++] Implement take kernel functions - properly handle special indicesARROW-2186 - [C++] Clean up architecture specific compiler flagsARROW-2217 - [C++] Add option to use dynamic linking for compression library dependenciesARROW-2298 - [Python] Add option to not consider NaN to be null when converting to an integer Arrow typeARROW-2412 - [Integration] Add nested dictionary integration testARROW-2467 - [Rust] Generate code using FlatbuffersARROW-2517 - [Java] Add list writer
ARROW-2618 - [Rust] Bitmap constructor should accept for flag for default state (0 or 1)ARROW-2667 - [C++/Python] Add pandas-like take method to ArrayARROW-2707 - [C++] Implement Table::Slice methods using Column::SliceARROW-2709 - [Python] write_to_dataset poor performance when splittingARROW-2730 - [C++] Set up CMAKE_C_FLAGS more thoughtfully instead of using CMAKE_CXX_FLAGSARROW-2796 - [C++] Simplify symbols.map file, use when building libarrow_pythonARROW-2818 - [Python] Better error message when passing SparseDataFrame into Table.from_pandasARROW-2835 - [C++] ReadAt/WriteAt are inconsistent with moving the files positionARROW-2969 - [R] Convert between StructArray and "nested" data.frame column containing data frame in each cellARROW-2981 - [C++] Support scripts / documentation for running clang-tidy on codebaseARROW-2984 - [JS] Refactor release verification script to share code with main source release verification scriptARROW-3040 - [Go] add support for comparing ArraysARROW-3041 - [Go] add support for TimeArrayARROW-3052 - [C++] Detect ORC system packagesARROW-3087 - [C++] Add kernels for comparison operations to scalarsARROW-3144 - [C++] Move "dictionary" member from DictionaryType to ArrayData to allow for changing dictionaries between Array chunksARROW-3150 - [Python] Ship Flight-enabled Python wheels on Linux and WindowsARROW-3166 - [C++] Consolidate IO interfaces used in arrow/io and parquet-cppARROW-3191 - [Java] Add support for ArrowBuf to point to arbitrary memory.ARROW-3200 - [C++] Add support for reading Flight streams with dictionariesARROW-3290 - [C++] Toolchain support for secure gRPCARROW-3294 - [C++] Test Flight RPC on Windows / AppveyorARROW-3314 - [R] Set -rpath using pkg-config when buildingARROW-3330 - [C++] Spawn multiple Flight performance servers in flight-benchmark to test parallel get performanceARROW-3419 - [C++] Run include-what-you-use checks as nightly buildARROW-3459 - [C++][Gandiva] Add support for variable length output vectorsARROW-3475 - [C++] Int64Builder.Finish(NumericArray)
ARROW-3570 - [Packaging] Don't bundle test data files with python wheelsARROW-3572 - [Packaging] Correctly handle ssh origin urls for crossbowARROW-3671 - [Go] implement Interval arrayARROW-3676 - [Go] implement Decimal128 arrayARROW-3679 - [Go] implement IPC protocolARROW-3680 - [Go] implement Float16 arrayARROW-3686 - [Python] Support for masked arrays in to/from numpyARROW-3702 - [R] POSIXct mapped to DateType not TimestampType?ARROW-3714 - [CI] Run RAT checks in pre-commit hooksARROW-3729 - [C++] Support for writing TIMESTAMP_NANOS Parquet metadataARROW-3732 - [R] Add functions to write RecordBatch or Schema to Message value, then read backARROW-3758 - [R] Build R library on Windows, document build instructions for Windows developersARROW-3759 - [R][CI] Build and test on Windows in AppveyorARROW-3767 - [C++] Add cast for Null to any typeARROW-3780 - [R] Failed to fetch data: invalid data when collecting int16ARROW-3791 - [C++] Add type inference for boolean values in CSV filesARROW-3794 - [R] Consider mapping INT8 to integer() not raw()ARROW-3804 - [R] Consider lowering required R runtimeARROW-3810 - [R] type= argument for Array and ChunkedArrayARROW-3811 - [R] struct arrays inferenceARROW-3814 - [R] RecordBatch$from_arrays()ARROW-3815 - [R] refine record batch factoryARROW-3848 - [R] allow nbytes to be missing in RandomAccessFile$Read()ARROW-3897 - [MATLAB] Add MATLAB support for writing numeric datatypes to a Feather fileARROW-3904 - [C++/Python] Validate scale and precision of decimal128 typeARROW-4013 - [Documentation][C++] Document how to build Apache Arrow on MSYS2ARROW-4020 - [Release] Remove source artifacts from dev dist system after release vote passesARROW-4047 - [Python] Document use of int96 timestamps and options in Parquet docsARROW-4086 - [Java] Add apis to debug alloc failuresARROW-4121 - [C++] Refactor memory allocation from InvertKernelARROW-4159 - [C++] Check for -Wdocumentation issuesARROW-4194 - [Format] Metadata.rst does not specify timezone for Timestamp typeARROW-4302 - [C++] Add OpenSSL to C++ build toolchainARROW-4337 - [C#] Array / RecordBatch Builder Fluent APIARROW-4343 - [C++] Add as complete as possible Ubuntu Trusty / 14.04 build to docker-compose setupARROW-4356 - [CI] Add integration (docker) test for turbodbcARROW-4369 - [Packaging] Release verification script should test linux packages via dockerARROW-4452 - [Python] Serializing sparse torch tensorsARROW-4453 - [Python] Create Cython wrappers for SparseTensorARROW-4467 - [Rust] [DataFusion] Create a REPL & Dockerfile for DataFusionARROW-4503 - [C#] ArrowStreamReader allocates and copies data excessivelyARROW-4504 - [C++] Reduce the number of unit test executablesARROW-4505 - [C++] Nicer PrettyPrint for date32ARROW-4566 - [C++][Flight] Add option to run arrow-flight-benchmark against a perf server running on a different hostARROW-4596 - [Rust] [DataFusion] Implement COUNT aggregate functionARROW-4622 - [C++] [Python] MakeDense and MakeSparse in UnionArray should accept a vector of FieldARROW-4625 - [Flight] Wrap server busy-wait methodsARROW-4626 - [Flight] Add application metadata field to DoGetARROW-4627 - [Flight] Add application metadata field to DoPutARROW-4701 - [C++] Add JSON chunker benchmarksARROW-4702 - [C++] Upgrade dependency versionsARROW-4708 - [C++] Add multithreaded JSON readerARROW-4708 - [C++] Add multithreaded JSON readerARROW-4714 - [C++][Java] Providing JNI interface to Read ORC file via Arrow C++ARROW-4717 - [C#] Consider exposing ValueTask instead of TaskARROW-4719 - [C#] Implement ChunkedArray, Column and Table in C#ARROW-4741 - [Java] Add documentation to all classes and enable checkstyle for class javadocsARROW-4787 - [C++] Include "null" values (perhaps with an option to toggle on/off) in hash kernel actionsARROW-4788 - [C++] Develop less verbose API for constructing StructArrayARROW-4800 - [C++] Create/port a StatusOr implementation to be able to return a status or a typeARROW-4805 - [Rust] Write temporal arrays to CSVARROW-4806 - [Rust] Support casting temporal arrays in cast kernelsARROW-4824 - [Python] read_csv should accept io.StringIO objectsARROW-4827 - [C++] Implement benchmark comparison between two git revisionsARROW-4847 - [Python] Add pyarrow.table factory function that dispatches to various ctors based on type of inputARROW-4904 - [C++] Move implementations in arrow/ipc/test-common.h into libarrow_testingARROW-4911 - [R] Support for building package for WindowsARROW-4912 - [C++, Python] Allow specifying column names to CSV readerARROW-4913 - [Java][Memory] Limit number of ledgers and arrowbufsARROW-4945 - [Flight] Enable Flight integration tests in TravisARROW-4956 - [C#] Allow ArrowBuffers to wrap external Memory in C#ARROW-4959 - [Gandiva][Crossbow] Builds brokenARROW-4968 - [Rust] StructArray builder and From<> methods should check that field types match schemaARROW-4971 - [Go] DataType equalityARROW-4972 - [Go] Array equalityARROW-4973 - [Go] Slice Array equalityARROW-4974 - [Go] Array approx equalityARROW-4990 - [C++] Kernel to compare array with arrayARROW-4993 - [C++] Display summary at the end of CMake configurationARROW-5000 - [Python] Fix deprecation warning from setup.pyARROW-5007 - [C++] Move DCHECK out of sse-utilsARROW-5020 - [C++][Gandiva] Split Gandiva-related conda packages for builds into separate .yml conda env fileARROW-5027 - [Python] Add JSON ReaderARROW-5037 - [Rust] [DataFusion] Refactor aggregate moduleARROW-5038 - [Rust] [DataFusion] Implement AVG aggregate functionARROW-5039 - [Rust] [DataFusion] Fix bugs in CAST supportARROW-5040 - [C++] ArrayFromJSON can't parse Timestamp from stringsARROW-5045 - [Rust] Code coverage silently failing in CIARROW-5053 - [Rust] [DataFusion] Use env var for location of arrow test dataARROW-5054 - [C++][Release] Test Flight in verify-release-candidate.shARROW-5056 - [Packaging] Adjust conda recipes to use ORC conda-forge package on unix systemsARROW-5061 - [Release] Improve 03-binary performanceARROW-5062 - [Java] Shade Java Guava dependency for FlightARROW-5063 - [Java] FlightClient should not create a child allocatorARROW-5064 - [Release] Pass PKG_CONFIG_PATH to glib in the verification scriptARROW-5066 - [Integration] Add flags to enable/disable implementations in integration/integration_test.pyARROW-5071 - [Benchmarking] Performs a benchmark run with archeryARROW-5076 - [Packaging] Improve post binary upload performanceARROW-5077 - [Rust] Release process should change Cargo.toml to use release versionsARROW-5078 - [Documentation] Sphinx is failed by RemovedInSphinx30WarningARROW-5079 - [Release] Add a script to release C# packageARROW-5080 - [Release] Add a script to release Rust packagesARROW-5081 - [C++] Consistently use PATH_SUFFIXES in CMake configARROW-5083 - [Developer] In merge_arrow_pr.py script, allow user to set a released Fix VersionARROW-5088 - [C++] Do not set -Werror when using BUILD_WARNING_LEVEL=CHECKIN in release modeARROW-5091 - [Flight] Rename FlightGetInfo message to FlightInfoARROW-5093 - [Packaging] Add support for selective binary uploadARROW-5094 - [Packaging] Add APT/Yum verification scriptsARROW-5102 - [C++] Reduce header dependenciesARROW-5108 - [Go] implement reading primitive arrays from Arrow fileARROW-5109 - [Go] implement reading binary/string arrays from Arrow fileARROW-5110 - [Go] implement reading struct arrays from Arrow fileARROW-5111 - [Go] implement reading list arrays from Arrow fileARROW-5112 - [Go] implement writing arrays to Arrow fileARROW-5113 - [C++][Flight] Unit tests in C++ for DoPutARROW-5115 - [JS] Implement the Vector BuildersARROW-5116 - [Rust] move kernel related files under compute/kernelsARROW-5124 - [C++] Add support for Parquet in MinGW buildARROW-5126 - [Rust] [Parquet] Convert parquet column desc to arrow data typeARROW-5127 - [Rust] [Parquet] Add page iteratorARROW-5136 - [Flight] Implement call options (timeouts)ARROW-5137 - [Flight] Implement authentication APIsARROW-5145 - [C++] Release mode lacks convenience input validationARROW-5150 - [Ruby] Add Arrow::Table#raw_recordsARROW-5155 - [GLib][Ruby] Add support for building union arrays from data typeARROW-5157 - [Website] Add MATLAB to powered by Apache Arrow pageARROW-5162 - [Rust] [Parquet] Rename mod reader to arrow.ARROW-5163 - [Gandiva] Cast timestamp/date are incorrectly evaluating year 0097 to 1997ARROW-5164 - [Gandiva] [C++] Introduce 32bit hash functionsARROW-5165 - [Python][Documentation] Build docs don't suggest assigning $ARROW_BUILD_TYPEARROW-5168 - [GLib] Add garrow_array_take()ARROW-5171 - [C++] Use LESS instead of LOWER in compare enum option.ARROW-5172 - [Go] implement reading fixed-size binary arrays from Arrow fileARROW-5178 - [Python] Allow creating Table from Python dictARROW-5179 - [Python] Return plain dicts, not OrderedDict, on Python 3.7+ARROW-5185 - [C++] Add support for Boost with CMake configuration fileARROW-5187 - [Rust] Ability to flatten StructArray into a RecordBatchARROW-5188 - [Rust] Add temporal builders for StructArrayARROW-5189 - [Rust] [Parquet] Format individual fields within a parquet rowARROW-5190 - [R] Discussion: tibble dependency in R packageARROW-5191 - [Rust] Expose CSV and JSON reader schemasARROW-5203 - [GLib] Add support for Compare filterARROW-5204 - [C++] Improve BufferBuilder performanceARROW-5212 - [Go] Array BinaryBuilder in Go library has no access to resize the values bufferARROW-5218 - [C++] Improve build when third-party library locations are specifiedARROW-5219 - [C++] Build protobuf_ep in parallel when using NinjaARROW-5222 - [Python] Issues with installing pyarrow for development on MacOSARROW-5225 - [Java] Improve performance of BaseValueVector#getValidityBufferSizeFromCountARROW-5226 - [Gandiva] support compare operators for decimalARROW-5238 - [Python] Improve usability of pyarrow.dictionary functionARROW-5241 - [Python] Add option to disable writing statistics to parquet fileARROW-5250 - [Java] remove javadoc suppression on methods.ARROW-5252 - [C++] Change variant implementationARROW-5256 - [Packaging][deb] Failed to build with LLVM 7.1.0ARROW-5257 - [Website] Update site to use "official" Apache Arrow logo, add clearly marked links to logoARROW-5258 - [C++/Python] Expose file metadata of dataset pieces to callerARROW-5261 - [C++] Finish implementation of scalar types for Duration and IntervalARROW-5262 - [Python] Fix typoARROW-5264 - [Java] Allow enabling/disabling boundary checking by environmental variableARROW-5266 - [Go] implement read/write IPC for Float16ARROW-5268 - [GLib] Add GArrowJSONReaderARROW-5269 - [C++] Whitelist benchmarks candidates for regression checksARROW-5275 - [C++] Write generic filesystem testsARROW-5281 - [Rust] [Parquet] Move DataPageBuilder to test_commonARROW-5284 - [Rust] Replace libc with std::alloc for memory allocationARROW-5286 - [Python] support Structs in Table.from_pandas given a known schemaARROW-5288 - [Documentation] Enrich the contribution guidelinesARROW-5289 - [C++] Move arrow/util/concatenate.h to arrow/array/ARROW-5290 - [Java] Provide a flag to enable/disable null-checking in vectors' get methodsARROW-5291 - [Python] Add wrapper for "take" kernel on ArrayARROW-5298 - [Rust] Add debug implementation for BufferARROW-5299 - [C++] ListArray comparison is incorrectARROW-5309 - [Python] Add clarifications to Python "append" methods that return new objectsARROW-5311 - [C++] Return more specific invalid Status in Take kernelARROW-5313 - [Format] Comments on Field table are a bit confusingARROW-5317 - [Rust] [Parquet] impl IntoIterator for SerializedFileReaderARROW-5319 - [CI] Enable ccache with MinGW buildsARROW-5321 - [Gandiva][C++] add isnull and isnotnull for utf8 and binary typesARROW-5323 - [CI] Use compression with clcacheARROW-5328 - [R] Add shell scripts to do a full package rebuild and test locallyARROW-5329 - Add support for building MATLAB interface to Feather directly within MATLABARROW-5334 - [C++] Add "Type" to names of arrow::Integer, arrow::FloatingPoint classes for consistencyARROW-5335 - [Python] Raise on variable dictionaries when converting to pandasARROW-5339 - [C++] Add jemalloc to thirdparty dependency download scriptARROW-5341 - [C++] Add instructions about fixing and testing for -Wdocumentation clang warnings locallyARROW-5342 - [Format] Formalize extension type metadata in IPC protocolARROW-5346 - [C++] Revert changes to qualify duration in vendored date codeARROW-5349 - [Python/C++] Provide a way to specify the file path in parquet ColumnChunkMetaDataARROW-5361 - [R] Follow DictionaryType/DictionaryArray changes from ARROW-3144ARROW-5363 - [GLib] Fix coding stylesARROW-5364 - [C++] Use ASCII rather than UTF-8 in BuildUtils.cmake commentARROW-5365 - [C++][CI] Add UBSan and ASAN into CIARROW-5368 - [C++] Disable jemalloc by default with MinGWARROW-5369 - [C++] Add support for glog on WindowsARROW-5370 - [C++] Detect system uriparser by defaultARROW-5372 - [GLib] Add support for null/boolean values CSV read optionARROW-5378 - [C++] Add local FileSystem implementationARROW-5384 - [Go] add FixedSizeList arrayARROW-5389 - [C++] Add an internal temporary directory APIARROW-5392 - [C++][CI][MinGW] Disable static library build on AppVeyorARROW-5393 - [R] Add tests and example for read_parquet()ARROW-5395 - [C++] Utilize stream EOS in File formatARROW-5396 - [JS] Ensure reader and writer support files and streams with no RecordBatchesARROW-5401 - [CI] [C++] Print ccache statistics on Travis-CIARROW-5404 - [C++] nonstd::string_view conflicts with std::string_view in c++17ARROW-5407 - [C++] Integration test Travis CI entry builds many unnecessary targetsARROW-5413 - [C++] CSV reader doesn't remove BOMARROW-5415 - [Release] Release script should update R version everywhereARROW-5416 - [Website] Add Homebrew to project installation pageARROW-5418 - [CI][R] Run code coverage and report to codecov.ioARROW-5420 - [Java] Implement or remove getCurrentSizeInBytes in VariableWidthVectorARROW-5427 - [Python] RangeIndex serialization change implicationsARROW-5428 - [C++] Add option to set "read extent" in arrow::io::BufferedInputStreamARROW-5429 - [Java] Provide alternative buffer allocation policyARROW-5432 - [Python] Add 'read_at' method to pyarrow.NativeFileARROW-5433 - [C++][Parquet] improve parquet-reader columns informationARROW-5434 - [Java] Introduce wrappers for backward compatibility for ArrowBuf changes in ARROW-3191ARROW-5436 - [Python] expose filters argument in parquet.read_tableARROW-5438 - [JS] Utilize stream EOS in File formatARROW-5441 - [C++] Implement FindArrowFlight.cmakeARROW-5442 - [Website] Clarify what makes a release artifact "official"ARROW-5443 - [Gandiva][Crossbow] Turn parquet encryption offARROW-5447 - [CI] [Ruby] CI is failed on AppVeyorARROW-5449 - [C++] Local filesystem implementation: investigate Windows UNC pathsARROW-5451 - [C++][Gandiva] Add round functions for decimalsARROW-5452 - [R] Add documentation website (pkgdown)ARROW-5461 - [Java] Add micro-benchmarks for Float8Vector and allocatorsARROW-5463 - [Rust] Implement AsRef for BufferARROW-5464 - [Archery] Bad --benchmark-filter defaultARROW-5465 - [Crossbow] Support writing submitted job definition yaml to a fileARROW-5466 - [Java] Dockerize Java builds in Travis CI, run multiple JDKs in single entryARROW-5467 - [Go] implement read/write IPC for Time32/Time64 arraysARROW-5468 - [Go] implement read/write IPC for Timestamp arraysARROW-5469 - [Go] implement read/write IPC for Date32/Date64 arraysARROW-5470 - [CI] C++ local filesystem patch breaks Travis R jobARROW-5472 - [Development] Add warning to PR merge tool if no JIRA component is setARROW-5474 - [C++] Document required Boost versionARROW-5475 - [Python] Add Python binding for arrow::ConcatenateARROW-5476 - [Java][Memory] Fix Netty ArrowBuf SliceARROW-5477 - [C++] Check required RapidJSON versionARROW-5478 - [Packaging] Drop Ubuntu 14.04 supportARROW-5481 - [GLib] garrow_seekable_input_stream_peek() misses "error" parameter documentARROW-5485 - [Gandiva][Crossbow] OSx builds failingARROW-5485 - [Gandiva][Crossbow] OSx builds failingARROW-5486 - [GLib] Add binding of gandiva::FunctionRegistry and related thingsARROW-5488 - [R] Workaround when C++ lib not availableARROW-5490 - [C++] Remove ARROW_BOOST_HEADER_ONLYARROW-5491 - [C++] Remove unecessary semicolons following MACRO definitionsARROW-5492 - [R] Add "col_select" argument to read_* functions to read subset of columnsARROW-5495 - [C++] Use HTTPS consistently for downloading dependenciesARROW-5496 - [R][CI] Fix relative paths in R codecov.io reportingARROW-5498 - [C++] Build failure with Flatbuffers 1.11.0 and MinGWARROW-5499 - [R] Alternate bindings for when libarrow is not foundARROW-5500 - [R] read_csv_arrow() signature should match readr::read_csv()ARROW-5503 - [R] add read_json()ARROW-5504 - [R] move use_threads argument to global optionARROW-5509 - [R] write_parquet()ARROW-5511 - [Packaging] Enable Flight in Conda packagesARROW-5512 - [C++] Draft initial public APIs for Datasets projectARROW-5513 - [Java] Refactor method name for getstartOffset to use camel caseARROW-5516 - [Python] Development page for pyarrow has a missing dependency in using pipARROW-5518 - [Java] Set VectorSchemaRoot rowCount to 0 on allocateNew and clearARROW-5524 - [C++] Turn off PARQUET_BUILD_ENCRYPTION in CMake if OpenSSL not foundARROW-5526 - [Developer] Add more prominent notice to GitHub issue template to direct bug reports to JIRAARROW-5529 - [Flight] Allow serving with multiple TLS certificatesARROW-5531 - [Python] Support binary, utf8, and nested types in Array.from_buffersARROW-5533 - [Plasma] Plasma client should be thread-safeARROW-5534 - [GLib] Add garrow_table_concatenate()ARROW-5535 - [GLib] Add garrow_table_slice()ARROW-5537 - [JS] Support delta dictionaries in RecordBatchWriter and DictionaryBuilderARROW-5538 - [C++] Restrict minimum OpenSSL version to 1.0.2ARROW-5541 - [R] cast from negative int32 to uint32 and uint64 are now safeARROW-5544 - [Archery] should not return non-zero in `benchmark diff` sub command on regressionARROW-5545 - [C++][Docs] Clarify expectation of UTC values for timestamps with time zones in C++ API docsARROW-5547 - [C++][FlightRPC] arrow-flight.pc isn't providedARROW-5552 - [Go] make Schema and Field implement StringerARROW-5554 - Add a python wrapper for arrow::ConcatenateARROW-5555 - [R] Add install_arrow() function to assist the user in obtaining C++ runtime librariesARROW-5556 - [Doc] Document JSON readerARROW-5557 - [C++] Investigate performance of VisitBitsUnrolled on different platformsARROW-5565 - [Python] Document how to use gdb when working on pyarrowARROW-5567 - [C++] Fix build error of memory-benchmarkARROW-5571 - [R] Rework handing of ARROW_R_WITH_PARQUETARROW-5574 - [R] documentation error for read_arrow()ARROW-5581 - [Java] Provide interfaces and initial implementations for vector sortingARROW-5582 - [Go] add support for comparing RecordsARROW-5586 - [R] convert Array of LIST type to R listsARROW-5587 - [Java] Add more maven style check for Java codeARROW-5590 - [R] Run "no libarrow" R build in the same CI entry if possibleARROW-5591 - [Go] implement read/write IPC for Duration & IntervalsARROW-5597 - [Packaging][deb] Add Flight packagesARROW-5600 - [R] R package namespace cleanupARROW-5602 - [Java][Gandiva] Add test for decimal round functionsARROW-5604 - [Go] improve test coverage of type-traitsARROW-5609 - [C++] Set CMP0068 CMake policy to avoid macOS warningsARROW-5612 - [Python][Documentation] Clarify date_as_object option behaviorARROW-5621 - [Go] implement read/write IPC for Decimal128 arraysARROW-5622 - [C++][Dataset] arrow-dataset.pc isn't providedARROW-5625 - [R] convert Array of struct type to data frame columnsARROW-5632 - [Doc] Add some documentation describing compile/debug workflow on macOS with Xcode IDEARROW-5633 - [Python] Enable bz2 in Linux wheelsARROW-5635 - [C++] Support "compacting" a tableARROW-5637 - [Gandiva] [Java]Complete IN ExpressionARROW-5639 - [Java] Remove floating point computation from getOffsetBufferValueCapacityARROW-5641 - [GLib] Remove enums files generated by GNU Autotools from Git targetsARROW-5643 - [Flight] Add ability to override hostname checkingARROW-5650 - [Python] Update manylinux dependency versionsARROW-5652 - [CI] Fix iwyu docker imageARROW-5653 - [CI] Fix cpp docker imageARROW-5656 - [Python] Enable Flight wheels on macOSARROW-5659 - [C++] Add support for finding OpenSSL installed by HomebrewARROW-5660 - [GLib][CI] Use the latest macOS image and all Homebrew based librariesARROW-5661 - Support hash functions for decimal in GandivaARROW-5662 - [C++] Add support for BOOST_SOURCE=AUTO|BUNDLED|SYSTEMARROW-5663 - [Packaging][RPM] Update CentOS packages for 0.14.0ARROW-5664 - [Crossbow] Execute nightly crossbow tests on CircleCI instead of TravisARROW-5668 - [Python] Display "not null" in Schema.__repr__ for non-nullable fieldsARROW-5669 - [Crossbow] manylinux1 wheel building failingARROW-5670 - [Crossbow] get_apache_mirror.py fails with TLS error on macOS with Python 3.5ARROW-5671 - [crossbow] mac os python wheels failingARROW-5672 - [Java] Refactor redundant method modifierARROW-5683 - [R] Add snappy to Rtools Windows buildsARROW-5684 - [Packaging][deb] Add support for Ubuntu 19.04ARROW-5685 - [Packaging][deb] Add support for Apache Arrow DatasetsARROW-5687 - [C++] Remove remaining uses of ARROW_BOOST_VENDOREDARROW-5690 - [Packaging][Python] macOS wheels broken: libprotobuf.18.dylib missingARROW-5694 - [Python] List of decimals are not supported when converting to pandasARROW-5695 - [C#][Release] Run sourcelink test in verify-release-candidate.shARROW-5696 - [Gandiva] [C++] Introduce castVarcharVarcharARROW-5699 - [C++] Optimize parsing of Decimal128 in CSVARROW-5701 - [C++][Gandiva] Build expressions only for the required selection vector typesARROW-5702 - [C++] parquet::arrow::FileReader::GetSchema()ARROW-5704 - [C++] Stop using ARROW_TEMPLATE_EXPORT for SparseTensorImpl classARROW-5705 - [Java] Optimize BaseValueVector#computeCombinedBufferSize logicARROW-5706 - [Java] Remove type conversion in getValidityBufferValueCapacityARROW-5707 - [Java] Improve the performance and code structure for ArrowRecordBatchARROW-5710 - [C++] Allow compiling Gandiva with Ninja on WindowsARROW-5715 - [Release] Verify Ubuntu 19.04 APT repositoryARROW-5718 - [R] auto splice data frames in record_batch() and table()ARROW-5720 - [C++] Create benchmarks for decimal related classes.ARROW-5721 - [Rust] Move array related code into a separate moduleARROW-5724 - [R] [CI] AppVeyor build should use ccacheARROW-5725 - [Crossbow] Port conda recipes to azure pipelinesARROW-5726 - [Java] Implement a common interface for int vectorsARROW-5727 - [Python] [CI] Install pytest-faulthandler before running testsARROW-5748 - [Packaging][deb] Add support for Debian GNU/Linux busterARROW-5749 - [Python] Add Python binding for Table::CombineChunks()ARROW-5751 - [Packaging][Python] Python macOS wheels have dynamic dependency on libcaresARROW-5752 - [Java] Improve the performance of ArrowBuf#setZeroARROW-5755 - [Rust] [Parquet] Add derived clone for TypeARROW-5768 - [Release] There are needless newlines at the end of CHANGELOG.mdARROW-5773 - [R] Clean up documentation before releaseARROW-5780 - [C++] Add benchmark for Decimal128 operationsARROW-5782 - [Release] Setup test data for Flight in dev/release/01-perform.shARROW-5783 - [Release][C#] Exclude dummy.git from RAT checkARROW-5785 - [Rust] Rust datafusion implementation should not depend on rustylineARROW-5787 - [Release][Rust] Use local modules to verify RCARROW-5793 - [Release] Avoid duplicate known host SSH error in dev/release/03-binary.shARROW-5794 - [Release] Skip uploading already uploaded binariesARROW-5795 - [Release] Add missing waits on uploading binariesARROW-5796 - [Release][APT] Update expected package listARROW-5797 - [Release][APT] Update supported distributionsARROW-5818 - [Java][Gandiva] support varlen output vectorsARROW-5820 - [Release] Remove undefined variable check from verify scriptARROW-5826 - [Website] Blog post for 0.14.0 release announcementPARQUET-1243 - [C++] Improve quality of error message for zero-length files, otherwise corrupted filesPARQUET-1411 - [C++] Upgrade to use LogicalType annotations instead of ConvertedTypePARQUET-1422 - [C++] Use Arrow IO interfaces natively rather than current parquet:: wrappersPARQUET-1517 - [C++] Update cpp crypto package to match signed-off specificationPARQUET-1523 - [C++] Vectorize comparator interfacePARQUET-1569 - [C++] Consolidate testing header filesPARQUET-1582 - [C++] Add ToString method ColumnDescriptorPARQUET-1583 - [C++] Remove parquet::Vector classPARQUET-1586 - [C++] Add --dump options to parquet-reader tool to dump def/rep levelsPARQUET-1603 - [C++] rename parquet::LogicalType to parquet::ConvertedTypeBug Fixes
ARROW-61 - [Java] Method can return the value bigger than long MAX_VALUEARROW-352 - [Format] Interval(DAY_TIME) has no unitARROW-1837 - [Java] Unable to read unsigned integers outside signed range for bit width in integration testsARROW-2119 - [C++][Java] Handle Arrow stream with zero record batchARROW-2136 - [Python] Non-nullable schema fields not checked in conversions from pandasARROW-2256 - [C++] Fuzzer builds fail out of the box on Ubuntu 16.04 using LLVM apt reposARROW-2461 - [Python] Build wheels for manylinux2010 tagARROW-2590 - [Python] Pyspark python_udf serialization error on grouped map (Amazon EMR)ARROW-3344 - [Python] test_plasma.py fails (in test_plasma_list)ARROW-3399 - [Python] Cannot serialize numpy matrix objectARROW-3650 - [Python] Mixed column indexes are read back as stringsARROW-3801 - [Python] Pandas-Arrow roundtrip makes pd categorical index not writeableARROW-4021 - [Ruby] Error building red-arrow on msys2ARROW-4076 - [Python] schema validation and filtersARROW-4139 - [Python] Cast Parquet column statistics to unicode if UTF8 ConvertedType is setARROW-4301 - [Java][Gandiva] Maven snapshot version update does not seem to update Gandiva submoduleARROW-4301 - [Java][Gandiva] Maven snapshot version update does not seem to update Gandiva submoduleARROW-4324 - [Python] Array dtype inference incorrect when created from list of mixed numpy scalarsARROW-4350 - [Python] dtype=object arrays cannot be converted to a list-of-list ListArrayARROW-4433 - [R] Segmentation fault when instantiating arrow::table from data frameARROW-4447 - [C++] Investigate dynamic linking for libthiftARROW-4516 - [Python] Error while creating a ParquetDataset on a path without `_common_dataset` but with an empty `_tempfile`ARROW-4523 - [JS] Add row proxy generation benchmarkARROW-4651 - [Format] Flight Location should be more flexible than a (host, port) pairARROW-4665 - [C++] With glog activated, DCHECK macros are redefinedARROW-4675 - [Python] Error serializing bool ndarray in py2 and deserializing in py3ARROW-4694 - [CI] detect-changes.py is inconsistentARROW-4723 - [Python] Skip _files when reading a directory containing parquet filesARROW-4725 - [C++] Dictionary tests disabled under MinGW buildsARROW-4823 - [Python] read_csv shouldn't close file handles it doesn't ownARROW-4832 - [Python] pandas Index metadata for RangeIndex is incorrectARROW-4845 - [R] Compiler warnings on Windows MingW64ARROW-4851 - [Java] BoundsChecking.java defaulting behavior for old drill parameter seems offARROW-4877 - [Plasma] CI failure in test_plasma_listARROW-4884 - [C++] conda-forge thrift-cpp package not available via pkg-config or cmakeARROW-4885 - [Python] read_csv() can't handle decimal128 columnsARROW-4886 - [Rust] Inconsistent behaviour with casting sliced primitive array to list arrayARROW-4923 - Expose setters for Decimal vector that take long and double inputsARROW-4934 - [Python] Address deprecation notice that will be a bug in Python 3.8ARROW-5019 - [C#] ArrowStreamWriter doesn't work on a non-seekable streamARROW-5049 - [Python] org/apache/hadoop/fs/FileSystem class not found when pyarrow FileSystem used in sparkARROW-5051 - [GLib][Gandiva] Test failure in release verification scriptARROW-5055 - [Ruby][MSYS2] libparquet needs to be installed in MSYS2 for rubyARROW-5058 - [Release] 02-source.sh generates e-mail template with wrong linksARROW-5059 - [C++][Gandiva] cbrt_* floating point tests can fail due to exact comparisonsARROW-5065 - [Rust] cast kernel does not support casting from Int64ARROW-5068 - [Gandiva][Packaging] Fix gandiva nightly builds after the CMake refactorARROW-5090 - Parquet linking fails on MacOS due to @rpath in dylibARROW-5092 - [C#] Source Link doesn't work with the C# release scriptARROW-5095 - [Flight][C++] Flight DoGet doesn't expose server error messageARROW-5096 - [Packaging][deb] plasma-store-server packages are missingARROW-5097 - [Packaging][CentOS6] arrow-lib has unresolvable dependenciesARROW-5098 - [Website] Update APT install document for 0.13.0ARROW-5100 - [JS] Writer swaps byte order if buffers share the same underlying ArrayBufferARROW-5117 - [Go] Panic when appending zero slices after initializing a builderARROW-5119 - [Go] invalid Stringer implementation for array.BooleanARROW-5122 - [Python] pyarrow.parquet.read_table raises non-file path error when given a windows path to a directoryARROW-5128 - [Packaging][CentOS][Conda] Numpy not found in nightly buildsARROW-5129 - [Rust][Parquet] Column writer bug: check dictionary encoder when adding a new data pageARROW-5130 - [Python] Segfault when importing TensorFlow after PyarrowARROW-5132 - [Java] Errors on building gandiva_jni.dll on Windows with Visual Studio 2017ARROW-5138 - [Python/C++] Row group retrieval doesn't restore index properlyARROW-5140 - [Bug?][Parquet] Can write a jagged array column of strings to disk, but hit `ArrowNotImplementedError` on readARROW-5142 - [CI] Fix conda calls in AppVeyor scriptsARROW-5144 - [Python] ParquetDataset and ParquetPiece not serializableARROW-5146 - [Dev] Merge script imposes directory nameARROW-5147 - [C++] get an error in building: Could NOT find DoubleConversionARROW-5148 - [CI] [C++] LLVM-related compile errorsARROW-5149 - [Packaging][Wheel] Pin LLVM to version 7 in windows buildsARROW-5152 - [Python] CMake warnings when buildingARROW-5159 - Unable to build benches in arrow crate.ARROW-5160 - [C++] ABORT_NOT_OK evalutes expression twiceARROW-5166 - [Python][Parquet] Statistics for uint64 columns may overflowARROW-5167 - [C++] Upgrade string-view-light to latestARROW-5169 - [Python] non-nullable fields are converted to nullable in {{Table.from_pandas}}ARROW-5173 - [Go] handle multiple concatenated streams back-to-backARROW-5174 - [Go] implement Stringer for DataTypesARROW-5177 - [Python] ParquetReader.read_column() doesn't check boundsARROW-5183 - [CI] MinGW build failures on AppVeyorARROW-5184 - [Rust] Broken links and other documentation warningsARROW-5186 - [Plasma] Crash on deleting CUDA memoryARROW-5194 - [C++][Plasma] TEST(PlasmaSerialization, GetReply) is failingARROW-5195 - [Python] read_csv ignores null_values on string typesARROW-5201 - [Python] Import ABCs from collections is deprecated in Python 3.7ARROW-5208 - [Python] Inconsistent resulting type during casting in pa.array() when mask is presentARROW-5214 - [C++] Offline dependency downloader misses some librariesARROW-5217 - [Rust] [CI] DataFusion test failureARROW-5232 - [Java] value vector size increases rapidly in case of clear/setSafe loopARROW-5233 - [Go] migrate to new flatbuffers-v1.11.0ARROW-5237 - [Python] pandas_version key in pandas metadata no longer populatedARROW-5240 - [C++][CI] cmake_format 0.5.0 appears to fail the buildARROW-5242 - [C++] Arrow doesn't compile cleanly with Visual Studio 2017 Update 9 or later due to narrowingARROW-5243 - [Java][Gandiva] Add test for decimal compare functionsARROW-5245 - [C++][CI] Unpin cmake_formatARROW-5246 - [Go] use Go-1.12 in CIARROW-5249 - [Java] Flight client doesn't handle auth correctly in some casesARROW-5253 - [C++] external Snappy fails on AlpineARROW-5254 - [Flight][Java] DoAction does not support result streamsARROW-5255 - [Java] Implement user-defined data types APIARROW-5260 - [Python][C++] Crash when deserializing from components in a fresh new processARROW-5274 - [JavaScript] Wrong array type for countByARROW-5283 - [C++][Plasma] Server crash when creating an aborted object 3 timesARROW-5285 - [C++][Plasma] GpuProcessHandle is not released when GPU object deletedARROW-5293 - [C++] Take kernel on DictionaryArray does not preserve ordered flagARROW-5294 - [CI] setuptools_scm failuresARROW-5296 - [Java] Sporadic Flight test failuresARROW-5301 - [Python] parquet documentation outdated on nthreads argumentARROW-5304 - [C++] CudaDeviceManager::GetInstance is not thread-safeARROW-5306 - [CI] [GLib] Disable GTK-DocARROW-5308 - [Go] remove deprecated Feather formatARROW-5314 - [Go] Incorrect Printing for String Arrays with OffsetsARROW-5314 - [Go] Incorrect Printing for String Arrays with OffsetsARROW-5318 - [Python] pyarrow hdfs reader overrequestsARROW-5325 - [Archery][Benchmark] Output properly formatted jsonlines from benchmark diff cli commandARROW-5330 - [Python] [CI] Run Python Flight tests on Travis-CIARROW-5332 - [R] R package fails to build/install: error in dyn.load()ARROW-5348 - [CI] [Java] Gandiva checkstyle failureARROW-5360 - [Rust] Builds are broken by rustyline on nightly 2019-05-16+ARROW-5362 - [C++] Compression round trip test can cause some sanitizers to to failARROW-5371 - [Release] Add tests for dev/release/00-prepare.shARROW-5373 - [Java] Add missing details for Gandiva Java BuildARROW-5376 - [C++] Compile failure on gcc 5.4.0ARROW-5383 - [Go] update IPC flatbuf (new Duration type)ARROW-5387 - [Go] properly handle sub-slice of ListARROW-5388 - [Go] use arrow.TypeEqual in array.NewChunkedARROW-5390 - [CI] Job time limit exceeded on TravisARROW-5397 - Test Flight TLS supportARROW-5398 - [Python] Flight tests broken by URI changesARROW-5403 - [C++] Test failures not propagated in Windows shared buildsARROW-5411 - [C++][Python] Build error building on Mac OS MojaveARROW-5412 - [Java] Integration test fails with UnsupportedOperationExceptionARROW-5419 - [C++] CSV strings_can_be_null option doesn't respect all null_valuesARROW-5421 - [Packaging][Crossbow] Duplicated key in nightly test configurationARROW-5422 - [CI] [C++] Build failure with Google BenchmarkARROW-5430 - [Python] Can read but not write parquet partitioned on large intsARROW-5435 - [Java] add test for IntervalYearVector#getAsStringBuilderARROW-5437 - [Python] Missing pandas pytest marker from parquet testsARROW-5446 - [C++] Use cmake header install directory instead of includeARROW-5448 - [CI] MinGW build failures on AppVeyorARROW-5453 - [C++] Just-released cmake-format 0.5.2 breaks the buildARROW-5455 - [Rust] Build broken by 2019-05-30 Rust nightlyARROW-5456 - [GLib][Plasma] Installed plasma-glib may be used on building documentARROW-5457 - [GLib][Plasma] Environment variable name for test is wrongARROW-5459 - [Go] implement Stringer for Float16 DataTypeARROW-5462 - [Go] support writing zero-length ListARROW-5479 - [Rust] [DataFusion] Use ARROW_TEST_DATA instead of relative path for testingARROW-5487 - [CI] [Python] Failure in docs buildARROW-5493 - [Integration/Go] add Go support for IPC integration testsARROW-5507 - [Plasma] [CUDA] Compile errorARROW-5514 - [C++] Printer for uint64 shows wrong valuesARROW-5517 - [C++] Header collection CMake logic should only consider filename without directory includedARROW-5520 - [C++][Packaging] No NVidia CUDA toolkit on AArch64CARROW-5521 - [Packaging] License check fails with Apache RAT 0.13ARROW-5528 - Concatenate() crashes when concatenating empty binary arrays.ARROW-5532 - [JS] Field Metadata Not ReadARROW-5551 - [Go] invalid FixedSizeArray representationARROW-5553 - [Ruby] red-arrow gem does not compile on ruby:2.5 docker imageARROW-5576 - [C++] Flaky thrift_ep tarball downloadsARROW-5577 - [C++] Link failure due to googletest shared library on Alpine LinuxARROW-5583 - [Java] When the isSet of a NullableValueHolder is 0, the buffer field should not be usedARROW-5584 - [Java] Add import for link reference in FieldReader javadocARROW-5589 - [C++][Fuzzing] arrow-ipc-fuzzing-test crash 2354085db0ARROW-5592 - [Go] implement Duration arrayARROW-5596 - [Python] Flight tests failing on Python 2.7ARROW-5601 - [gandiva] Error when projector with a string fieldARROW-5603 - [Python] register pytest markers to avoid warningsARROW-5605 - [C++][Fuzzing] arrow-ipc-fuzzing-test crash 74aec871d1ARROW-5606 - [Python] pandas.RangeIndex._start/_stop/_step are deprecatedARROW-5608 - [C++][parquet] Invalid memory access when using parquet::arrow::ColumnReaderARROW-5615 - [C++] Compilation error due to C++11 string literals on gcc 5.4.0 Ubuntu 16.04ARROW-5616 - [Python] C++ build failure against Python 2.7 headersARROW-5617 - [C++] thrift_ep 0.12.0 fails to build when using ARROW_BOOST_VENDORED=ONARROW-5619 - [C++] get_apache_mirror.py doesn't work with Python 3.5ARROW-5623 - [CI][GLib] Failed on macOSARROW-5624 - [C++] -Duriparser_SOURCE=BUNDLED is brokenARROW-5626 - [C++][Gandiva] Expression cache should consider precision and scale tooARROW-5629 - [C++] Fix Coverity issuesARROW-5631 - [C++] CMake 3.2 build is brokenARROW-5644 - [Python] test_flight.py::test_tls_do_get appears to hangARROW-5647 - [Python] Accessing a file from Databricks using pandas read_parquet using the pyarrow engine fails with : Passed non-file path: /mnt/aa/example.parquetARROW-5648 - [C++] Build fails on mingw without codecvtARROW-5654 - [C++] ChunkedArray should validate the types of the arraysARROW-5657 - [C++] "docker-compose run cpp" broken in masterARROW-5674 - [Python] Missing pandas pytest markers from test_parquet.pyARROW-5675 - [Doc] Fix typo in documentation describing compile/debug workflow on macOS with Xcode IDEARROW-5678 - [R][Lint] Fix hadolint docker linting errorARROW-5693 - [Go] skip IPC integration test for Decimal128ARROW-5697 - [GLib] c_glib/Dockerfile is brokenARROW-5698 - [R] r/Dockerfile docker-compose build is brokenARROW-5709 - [C++] gandiva-date_time_test failure on WindowsARROW-5714 - [JS] Inconsistent behavior in Int64Builder with/without BigNumARROW-5723 - [Gandiva][Crossbow] Builds failingARROW-5728 - [Python] [CI] Travis-CI failures in test_jvm.pyARROW-5729 - [Python][Java] ArrowType.Int object has no attribute 'isSigned'ARROW-5730 - [Python][CI] Selectively skip test cases in the dask integration testARROW-5732 - [C++] macOS builds failing idiosyncratically on master with warnings from pmmintrin.hARROW-5735 - [C++] Appveyor builds failing persistently in thrift_ep buildARROW-5737 - [C++][Gandiva] Gandiva not building in manylinuxARROW-5738 - [Crossbow][Conda] OSX package builds are failing with missing intrinsicsARROW-5739 - [CI] Fix docker python buildARROW-5750 - [Java] Java compilation failures on masterARROW-5754 - [C++]Missing override for ~GrpcStreamWriter?ARROW-5765 - [C++] TestDictionary.Validate test is crashed with release buildARROW-5769 - [Java] org.apache.arrow.flight.TestTls is failed via dev/release/00-prepare.shARROW-5770 - [C++] Fix -Wpessimizing-move in result.hARROW-5771 - [Python] Docker python-nopandas job failsARROW-5774 - [Java][Documentation] Document the need to checkout git submodules for flightARROW-5781 - [Archery] Ensure benchmark clone accepts remotes in revisionARROW-5791 - [Python] pyarrow.csv.read_csv hangs + eats all RAMARROW-5816 - [Release] Parallel curl does not work reliably in verify-release-candidate-shARROW-5922 - [Python] Unable to connect to HDFS from a worker/data node on a Kerberized cluster using pyarrow' hdfs APIPARQUET-1402 - [C++] incorrect calculation column start offset for files created by parquet-mr 1.8.1PARQUET-1405 - [C++] 'Couldn't deserialize thrift' error when reading large binary columnPARQUET-1405 - [C++] 'Couldn't deserialize thrift' error when reading large binary columnPARQUET-1565 - [C++] SEGV in FromParquetSchema with corrupt file from PARQUET-1481PARQUET-1571 - [C++] Can't read data from parquet file in C++ libraryPARQUET-1574 - [C++] parquet-encoding-test failed with msvcPARQUET-1581 - [C++] Fix undefined behavior in encoding.cc when num_dictionary_values is 0.
Apache Arrow 0.13.0 (2019-04-01)
Bug Fixes
ARROW-295 - Create DOAP FileARROW-1171 - [C++] Segmentation faults on Fedora 24 with pyarrow-manylinux1 and self-compiled turbodbcARROW-2392 - [Python] pyarrow RecordBatchStreamWriter allows writing batches with different schemasARROW-2399 - [Rust] Builder should not provide a set() method
ARROW-2598 - [Python] table.to_pandas segfaultARROW-3086 - [GLib] GISCAN fails due to conda-shipped openblasARROW-3096 - [Python] Update Python source build instructions given Anaconda/conda-forge toolchain migrationARROW-3133 - [C++] Logical boolean kernels in kernels/boolean.cc cannot write into preallocated memoryARROW-3133 - [C++] Logical boolean kernels in kernels/boolean.cc cannot write into preallocated memoryARROW-3208 - [C++] Segmentation fault when casting dictionary to numeric with nullptr valid_bitmapARROW-3426 - [CI] Java integration test very verboseARROW-3564 - [Python] writing version 2.0 parquet format with dictionary encoding enabledARROW-3578 - [Release] Address spurious Apache RAT failures in source release scriptARROW-3593 - [R] CI builds failing due to GitHub API rate limitsARROW-3606 - [Python] flake8 fails on CrossbowARROW-3669 - [Python] Convert big-endian numbers or raise error in pyarrow.arrayARROW-3843 - [Python] Writing Parquet file from empty table created with Table.from_pandas(..., preserve_index=False) failsARROW-3923 - [Java] JDBC-to-Arrow Conversion: Unnecessary Calendar RequirementARROW-4007 - [Java][Plasma] Plasma JNI tests failingARROW-4050 - [Python][Parquet] core dump on reading parquet fileARROW-4081 - [Go] Sum methods on Mac OS X panic when the array is emptyARROW-4104 - [Java] race in AllocationManager during releaseARROW-4108 - [Python/Java] Spark integration tests do not workARROW-4117 - [Python] "asv dev" command fails with latest revisionARROW-4140 - [C++][Gandiva] Compiled LLVM bitcode file path may result in libraries being non-relocatableARROW-4145 - [C++] Find Windows-compatible strptime implementationARROW-4181 - [Python] TestConvertStructTypes.test_from_numpy_large failingARROW-4192 - "./dev/run_docker_compose.sh" is out of dateARROW-4213 - [Flight] C++ and Java implementations are incompatibleARROW-4244 - Clarify language around padding/alignmentARROW-4250 - [C++][Gandiva] Use approximate comparisons for floating point numbers in gandiva-projector-testARROW-4252 - [C++] Status error context strings missing lines of codeARROW-4253 - [GLib] Cannot use non-system Boost specified with $BOOST_ROOTARROW-4254 - [C++] Gandiva tests fail to compile with Boost in Ubuntu 14.04 aptARROW-4255 - [C++] Schema::GetFieldIndex is not thread-safeARROW-4261 - [C++] CMake paths for IPC, Flight, Thrift, and Plasma don't support using Arrow as a subprojectARROW-4264 - [C++] Document why DCHECKs are used in kernelsARROW-4267 - [Python/C++][Parquet] Segfault when reading rowgroups with duplicated columnsARROW-4274 - [Gandiva] static jni library broken after decimal changesARROW-4275 - [C++] gandiva-decimal_single_test extremely slowARROW-4280 - [C++][Documentation] It looks like flex and bison are required for parquetARROW-4282 - [Rust] builder benchmark is brokenARROW-4284 - [C#] File / Stream serialization fails due to type mismatch / missing footerARROW-4295 - [Plasma] Incorrect log message when evicting objectsARROW-4296 - [Plasma] Starting Plasma store with use_one_memory_mapped_file enabled crashes due to improper memory alignmentARROW-4308 - [Python] pyarrow has a hard dependency on pandasARROW-4311 - [Python] Regression on pq.ParquetWriter incorrectly handling source stringARROW-4312 - [C++] Lint doesn't work anymore ("[Errno 24] Too many open files")ARROW-4319 - plasma/store.h pulls ins flatbuffer dependencyARROW-4320 - [C++] Add tests for non-contiguous tensorsARROW-4322 - [CI] docker nightlies fails after conda-forge compiler migrationARROW-4323 - [Packaging] Fix failing OSX clang conda forge buildsARROW-4326 - [C++] Development instructions in python/development.rst will not work for many Linux distros with new conda-forge toolchainARROW-4327 - [Python] Add requirements-build.txt file to simplify setting up Python build environmentARROW-4328 - Make R build compatible with DARROW_TENSORFLOW=ONARROW-4329 - Python should include the parquet headersARROW-4342 - [Gandiva][Java] spurious failures in projector cache testARROW-4347 - [Python] Run Python Travis CI unit tests on Linux when Java codebase changedARROW-4349 - [C++] Build all benchmarks on Windows without failingARROW-4351 - [C++] Fail to build with static parquetARROW-4355 - [C++] test-util functions are no longer part of libarrowARROW-4360 - [C++] Query homebrew for ThriftARROW-4364 - [C++] Fix -weverything -wextra compilation errorsARROW-4366 - [Docs] Change extension from format/README.md to format/README.rstARROW-4367 - [C++] StringDictionaryBuilder segfaults on Finish with only null entriesARROW-4368 - Bintray repository signature verification failsARROW-4370 - [Python] Table to pandas conversion fails for list of boolARROW-4374 - [C++] DictionaryBuilder does not correctly report length and null_countARROW-4381 - [Docker] docker-compose build lint failsARROW-4382 - [C++] Improve new cpplint output readabilityARROW-4384 - [C++] Running "format" target on new Windows 10 install opens "how do you want to open this file" dialogARROW-4385 - [Python] default_version of a release should not include SNAPSHOTARROW-4389 - [R] Installing clang-tools in CI is failing on trustyARROW-4395 - ts-node throws type error running `bin/arrow2csv.js`ARROW-4400 - [CI] install of clang tools failingARROW-4403 - [Rust] CI fails due to formatting errorsARROW-4404 - [CI] AppVeyor toolchain build does not build anythingARROW-4407 - [C++] ExternalProject_Add does not capture CC/CXX correctlyARROW-4410 - [C++] Fix InvertKernel edge casesARROW-4413 - [Python] pyarrow.hdfs.connect() failingARROW-4414 - [C++] Stop using cmake COMMAND_EXPAND_LISTS because it breaks package builds for older distrosARROW-4417 - [C++] Doc build brokenARROW-4420 - [INTEGRATION] Make spark integration test pass and test against spark's master branchARROW-4421 - [Flight][C++] Handle large Flight data messagesARROW-4434 - [Python] Cannot create empty StructArray via pa.StructArray.from_arraysARROW-4440 - [C++] Fix flatbuffers build using msvcARROW-4457 - [Python] Cannot create Decimal128 array using integersARROW-4469 - [Python][C++] CI Failing for Python 2.7 and 3.6 with valgrindARROW-4471 - [C++] Pass AR and RANLIB to all external projectsARROW-4474 - [Flight] FlightInfo should use signed integer types for payload sizeARROW-4480 - [Python] Drive letter removed when writing parquet fileARROW-4487 - [C++] Appveyor toolchain build does not actually build the projectARROW-4494 - [Java] arrow-jdbc JAR is not uploaded on releaseARROW-4496 - [CI] CI failing for python Xcode 7.3ARROW-4498 - [Plasma] Plasma fails building with CUDA enabledARROW-4500 - [C++] librt and pthread hacks can cause linking problemsARROW-4501 - [C++] Unique returns non-unique stringsARROW-4525 - [Rust] [Parquet] Convert ArrowError to ParquetErrorARROW-4527 - [Packaging] Update linux packaging tasks to align with the LLVM 7 migrationARROW-4532 - [Java] varchar value buffer much larger than expectedARROW-4533 - [Python] Document how to run hypothesis testsARROW-4535 - [C++] Fix MakeBuilder to preserve ListType's field nameARROW-4536 - Add data_type argument in garrow_list_array_newARROW-4538 - [PYTHON] Remove index column from subschema in write_to_dataframeARROW-4549 - [C++] Can't build benchmark code on CUDA enabled buildARROW-4550 - [JS] Fix AMD patternARROW-4559 - [Python] pyarrow can't read/write filenames with special charactersARROW-4563 - [Python] pa.decimal128 should validate inputsARROW-4571 - [Format] Tensor.fbs file has multiple root_type declarationsARROW-4573 - [Python] Add Flight unit testsARROW-4576 - [Python] Benchmark failuresARROW-4577 - [C++] Interface link libraries declared on arrow_shared target that are actually non-interfaceARROW-4581 - [C++] gbenchmark_ep is a dependency of unit tests when ARROW_BUILD_BENCHMARKS=ONARROW-4582 - [C++/Python] Memory corruption on Pandas->Arrow conversionARROW-4584 - [Python] Add built wheel to manylinux1 dockerignore.ARROW-4585 - [C++] Dependency of Flight C++ sources on generated protobuf is not respectedARROW-4587 - Flight C++ DoPut segfaultsARROW-4597 - [C++] Targets for system Google Mock shared library are missingARROW-4601 - [Python] Master build is broken due to missing licence for .dockerignoreARROW-4606 - [Rust] [DataFusion] FilterRelation created RecordBatch with empty schemaARROW-4608 - [C++] cmake script assumes that double-conversion installs static libsARROW-4617 - [C++] Support double-conversion<3.1ARROW-4624 - [C++] Linker errors when building benchmarksARROW-4629 - [Python] Pandas to arrow conversion slowed down by local importsARROW-4635 - [Java] StructVector growing validity buffer unnecessarilyARROW-4639 - [CI] Crossbow build failing for Gandiva jarsARROW-4641 - [C++] Flight builds complain of -Wstrict-aliasingARROW-4642 - [R] Change `f` to `file` in `read_parquet_file()`ARROW-4653 - [C++] decimal multiply broken when both args are negativeARROW-4654 - [C++] Implicit Flight target dependencies cause compilation failureARROW-4657 - [Release] gbenchmark should not be needed for verificationARROW-4658 - [C++] Shared gflags is also a run-time conda requirementARROW-4659 - [CI] ubuntu/debian nightlies fail because of missing gandiva filesARROW-4660 - [C++] gflags fails to build due to CMake errorARROW-4664 - [C++] DCHECK macro conditions are evaluated in release buildsARROW-4669 - [Java] No Bounds checking on ArrowBuf.sliceARROW-4672 - [C++] clang-7 matrix entry is build using gccARROW-4680 - [CI] [Rust] Travis CI builds fail with latest Rust 1.34.0-nightly (2019-02-25)ARROW-4684 - [Python] CI failures in test_cython.pyARROW-4687 - [Python] FlightServerBase.run should exit on Ctrl-CARROW-4688 - [C++][Parquet] 16MB limit on (nested) column chunk prevents tuning row_group_sizeARROW-4696 - Verify release script is over optimist with CUDA detectionARROW-4699 - [C++] json parser should not rely on null terminated buffersARROW-4704 - [CI][GLib] Plasma test is flakyARROW-4710 - [C++][R] New linting script skip files with "cpp" extensionARROW-4712 - [C++][CI] Clang7 Valgrind complains when not move shared_ptrARROW-4721 - [Rust] [DataFusion] Propagate schema in filterARROW-4724 - [C++] Python not being built nor test under MinGW buildsARROW-4728 - [JS] Failing test Table#assign with a zero-length Null column round-trips through serializationARROW-4737 - [C#] tests are not running in CIARROW-4744 - [CI][C++] Mingw32 builds failingARROW-4750 - [C++] RapidJSON triggers Wclass-memaccess on GCC 8+ARROW-4760 - [C++] protobuf 3.7 defines EXPECT_OK that clashes with Arrow's macroARROW-4766 - [C++] Casting empty boolean array causes segfaultARROW-4767 - [C#] ArrowStreamReader crashes while reading the end of a streamARROW-4768 - [C++][CI] arrow-test-array sometimes gets stuck in MinGW buildARROW-4774 - [C++][Parquet] Call Table::Validate when writing a tableARROW-4775 - [Website] Site navbar cannot be expandedARROW-4783 - [C++][CI] Mingw32 builds sometimes timeoutARROW-4793 - [Ruby] Suppress unused variable warningARROW-4796 - [Flight][Python] segfault in simple server implementationARROW-4802 - [Python] Hadoop classpath discovery broken HADOOP_HOME is a symlinkARROW-4807 - [Rust] Fix csv_writer benchmarkARROW-4811 - [C++] An incorrect dependency leads "ninja" to re-evaluate steps unnecessarily on subsequent callsARROW-4813 - [Ruby] Add tests for #== and #!=ARROW-4820 - [Python] hadoop class path derived not correctARROW-4822 - [C++/Python] pyarrow.Table.equals segmentation fault on NoneARROW-4828 - [Python] manylinux1 docker-compose context should be python/manylinux1ARROW-4850 - [CI] Integration test failures do not fail the Travis CI buildARROW-4853 - [Rust] Array slice doesn't work on ListArray and StructArrayARROW-4857 - [C++/Python/CI] docker-compose in manylinux1 crossbow jobs too oldARROW-4866 - [C++] zstd ExternalProject failing on WindowsARROW-4867 - [Python] Table.from_pandas() column order not respectedARROW-4869 - [C++] Use of gmock fails in compute/kernels/util-internal-test.ccARROW-4870 - [Ruby] gemspec has wrong msys2 dependency listedARROW-4871 - [Flight][Java] Handle large Flight messagesARROW-4872 - [Python] Keep backward compatibility for ParquetDatasetPieceARROW-4879 - [C++] cmake can't use conda's flatbuffersARROW-4881 - [Python] bundle_zlib CMake function still uses ARROW_BUILD_TOOLCHAINARROW-4900 - mingw-w64 < 5 does not have __cpuidexARROW-4903 - [C++] Building tests using only static libs not possibleARROW-4906 - [Format] Fix document to describe that SparseMatrixIndexCSR assumes indptr is sorted for each rowARROW-4918 - [C++] Add cmake-format to pre-commitARROW-4928 - [Python] Hypothesis test failuresARROW-4931 - [C++] CMake fails on gRPC ExternalProjectARROW-4938 - [Glib] Undefined symbols error occurred when GIR file is being generated.ARROW-4942 - [Ruby] Remove needless omitsARROW-4948 - [JS] Nightly test failing with "Cannot assign to read only property"ARROW-4950 - [C++] Thirdparty CMake error get_target_property() called with non-existent target LZ4::lz4ARROW-4952 - [C++] Equals / ApproxEquals behaviour undefined on FP NaNsARROW-4953 - [Ruby] Not loading libarrow-glibARROW-4954 - [Python] test failure with Flight enabledARROW-4958 - [C++] Purely static linking brokenARROW-4961 - [C++][Python] Add GTest_SOURCE=BUNDLED to relevant build docs that use conda-forge toolchainARROW-4962 - [C++] Warning level to CHECKIN can't compile on modern GCCARROW-4976 - [JS] RecordBatchReader should reset its Node/DOM streamsARROW-4982 - [GLib][CI] Run tests on AppVeyorARROW-4984 - [Flight][C++] Flight server segfaults when port is in useARROW-4986 - [CI] Travis fails to install llvm@7ARROW-4989 - [C++] Builds fails to find Ubuntu-packaged re2 libraryARROW-4991 - [CI] Bump travis node version to 11.12ARROW-4997 - [C#] ArrowStreamReader doesn't consume whole stream and doesn't implement sync readARROW-5009 - [C++] Cleanup using to std::* in filesARROW-5010 - [Release] Fix release script with llvm-7ARROW-5012 - [C++] "testing" headers not installedARROW-5023 - [Release] Default value syntax in shell is wrongARROW-5024 - [Release] crossbow.py --arrow-version causes missing variable errorARROW-5025 - [Python][Packaging] wheel for Windows are brokenARROW-5026 - [Python][Packaging] conda package on non Windows is brokenARROW-5029 - [C++] Compilation warnings in release modeARROW-5031 - [Dev] Release verification script does not run CUDA tests in PythonARROW-5042 - [Release] Wrong ARROW_DEPENDENCY_SOURCE in verification scriptARROW-5043 - [Release][Ruby] red-arrow dependency can't be resolve in verification scriptARROW-5044 - [Release][Rust] Format error in verification scriptARROW-5046 - [Release][C++] Plasma test is fragile in verification scriptARROW-5047 - [Release] Always set up parquet-testing in verification scriptARROW-5048 - [Release][Rust] arrow-testing is missing in verification scriptARROW-5050 - [C++] cares_ep should build before grpc_epARROW-5087 - [Debian] APT repository no longer contains libarrow-devARROW-5658 - [JAVA] Provide ability to resync VectorSchemaRoot if types changePARQUET-1482 - [C++] Unable to read data from parquet file generated with parquetjsPARQUET-1494 - [C++] Can't access parquet statistics on binary columnsPARQUET-1532 - [C++] Can't build column reader test with MinGWNew Features and Improvements
ARROW-47 - [C++] Consider adding a scalar type object modelARROW-331 - [Python] Timeline for dropping Python 2.7 supportARROW-549 - [C++] Add function to concatenate like-typed arraysARROW-572 - [C++] Apply visitor pattern in IPC metadataARROW-585 - [C++] Define public API for user-defined data typesARROW-694 - [C++] Build JSON "scanner" for reading record batches from line-delimited JSON filesARROW-1425 - [Python] Document semantic differences between Spark timestamps and Arrow timestampsARROW-1572 - [C++] Implement "value counts" kernels for tabulating value frequenciesARROW-1639 - [Python] More efficient serialization for RangeIndex in serialize_pandasARROW-1642 - [GLib] Build GLib using Meson in AppveyorARROW-1807 - [JAVA] Reduce Heap Usage (Phase 3): consolidate buffersARROW-1896 - [C++] Do not allocate memory for primitive outputs in CastKernel::Call implementationARROW-2015 - [Java] Use Java Time and Date APIs instead of JodaTimeARROW-2022 - [Format] Add custom metadata field specific to a RecordBatch messageARROW-2112 - [C++] Enable cpplint to be run on WindowsARROW-2243 - [C++] Enable IPO/LTOARROW-2409 - [Rust] Test for build warnings, remove current warningsARROW-2460 - [Rust] Schema and DataType::Struct should use Vec>
ARROW-2487 - [C++] Provide a variant of AppendValues that takes bytemaps for the nullabilityARROW-2523 - [Rust] Implement CAST operations for arraysARROW-2620 - [Rust] Integrate memory pool abstraction with rest of codebaseARROW-2627 - [Python] Add option (or some equivalent) to toggle memory mapping functionality when using parquet.ParquetFile or other read entry pointsARROW-2904 - [C++] Use FirstTimeBitmapWriter instead of SetBit functions in builder.h/ccARROW-3066 - [Wiki] Add "How to contribute" to developer wikiARROW-3084 - [Python] Do we need to build both unicode variants of pyarrow wheels?ARROW-3107 - [C++] arrow::PrettyPrint for Column instancesARROW-3121 - [C++] Mean kernel aggregateARROW-3123 - [C++] Incremental Count, Count Not Null aggregatorARROW-3135 - [C++] Add helper functions for validity bitmap propagation in kernel contextARROW-3149 - [C++] Use gRPC (when it exists) from conda-forge for CI buildsARROW-3162 - [Python] Enable Flight servers to be implemented in pure PythonARROW-3162 - [Python] Enable Flight servers to be implemented in pure PythonARROW-3239 - [C++] Improve random data generation functionsARROW-3255 - [C++/Python] Migrate Travis CI jobs off Xcode 6.4ARROW-3289 - [C++] Implement DoPut command for Flight on client and server sideARROW-3292 - [C++] Test Flight RPC in Travis CIARROW-3295 - [Packaging] Package gRPC libraries in conda-forge for use in builds, packagingARROW-3297 - [Python] Python bindings for Flight C++ clientARROW-3311 - [R] Functions for deserializing IPC components from arrow::Buffer or from IO interfaceARROW-3328 - [Flight] Allow for optional unique flight identifier to be sent with FlightGetInfoARROW-3361 - [R] Run cpp/build-support/cpplint.py on C++ source filesARROW-3364 - [Doc] Document docker compose setupARROW-3367 - [INTEGRATION] Port Spark integration test to the docker-compose setupARROW-3422 - [C++] Add "toolchain" target to ensure that all required toolchain libraries are builtARROW-3434 - [Packaging] Add Apache ORC C++ library to conda-forgeARROW-3435 - [C++] Add option to use dynamic linking with re2ARROW-3511 - [Gandiva] support input selection vectors for both projector and filterARROW-3532 - [Python] Schema, StructType, StructArray field retrieval by name should raise warning or exception for multiple matchesARROW-3550 - [C++] Use kUnknownNullCount in NumericArray constructorARROW-3554 - [C++] Reverse traits for C++ARROW-3594 - [Packaging] Build "cares" library in conda-forgeARROW-3595 - [Packaging] Build boringssl in conda-forgeARROW-3596 - [Packaging] Build gRPC in conda-forgeARROW-3619 - [R] Expose global thread pool optinsARROW-3631 - [C#] Add Appveyor build for C#ARROW-3653 - [Python/C++] Support data copying between different GPU devicesARROW-3735 - [Python] Proper error handling in _ensure_typeARROW-3761 - [R] Bindings for CompressedInputStream, CompressedOutputStreamARROW-3763 - [C++] Write Parquet ByteArray / FixedLenByteArray reader batches directly into arrow::BinaryBuilderARROW-3769 - [C++] Support reading non-dictionary encoded binary Parquet columns directly as DictionaryArrayARROW-3770 - [C++] Validate or add option to validate arrow::Table schema in parquet::arrow::FileWriter::WriteTableARROW-3816 - [R] nrow.RecordBatch methodARROW-3824 - [R] Document developer workflow for building project, running unit tests in r/README.mdARROW-3838 - [Rust] Implement CSV WriterARROW-3846 - [Gandiva] Build on WindowsARROW-3882 - [Rust] PrimitiveArray should support cast operations
ARROW-3903 - [Python] Random array generator for Arrow conversion and Parquet testingARROW-3926 - [Python] Add Gandiva bindings to Python wheelsARROW-3951 - [Go] implement a CSV writerARROW-3954 - [Rust] Add Slice to Array and ArrayDataARROW-3965 - [Java] JDBC-to-Arrow Conversion: Configuration ObjectARROW-3966 - [Java] JDBC-to-Arrow Conversion: JDBC Metadata in Schema FieldsARROW-3972 - [C++] Update to LLVM and Clang bits to 7.0ARROW-3981 - [C++] Rename json.hARROW-3985 - [C++] Pass -C option when compiling with ccache to avoid some warningsARROW-4012 - [Documentation][C++] Document how to install Apache Arrow on MSYS2ARROW-4014 - [C++] Fix "LIBCMT" warnings on MSVCARROW-4023 - [Gandiva] Address long CI times in macOS buildsARROW-4024 - [Python] Cython compilation error on cython==0.27.3ARROW-4031 - [C++] Refactor ArrayBuilder bitmap logic into TypedBufferBuilder
ARROW-4040 - [Rust] Add array_ops method for filtering an arrayARROW-4056 - [C++] Upgrade to boost-cpp 1.69.0 againARROW-4061 - [Rust] [Parquet] Implement "spaced" version for non-dictionary encoding/decodingARROW-4068 - [Gandiva] Support building with Xcode 6.4ARROW-4071 - [Rust] Add rustfmt as a pre-commit hookARROW-4072 - [Rust] Set default value for PARQUET_TEST_DATAARROW-4092 - [Rust] Implement common Reader / DataSource trait for CSV and ParquetARROW-4094 - [Python] Store RangeIndex in Parquet files as metadata rather than a physical data columnARROW-4110 - [C++] Do not generate distinct cast kernels when input and output type are the sameARROW-4123 - [C++] Improve linting workflow and documentation for Windows-based developersARROW-4124 - [C++] Abstract aggregation kernel APIARROW-4142 - [Java] JDBC-to-Arrow: JDBC ArraysARROW-4165 - [C++] Port cpp/apidoc/Windows.md and other files to Sphinx / rstARROW-4180 - [Java] Reduce verbose logging of ArrowBuf creation events?ARROW-4196 - [Rust] Add explicit SIMD vectorization for arithmetic ops in "array_ops"ARROW-4198 - [Gandiva] Add support to cast timestampARROW-4204 - [Gandiva] implement decimal subtractARROW-4205 - [Gandiva] Implement decimal multiplyARROW-4206 - [Gandiva] Implement decimal divideARROW-4212 - [Python] [CUDA] Creating a CUDA buffer from Numba device array should be easierARROW-4230 - [C++] Enable building flight against system gRPCARROW-4232 - [C++] Follow conda-forge compiler ABI migrationARROW-4234 - [C++] Add memory bandwidth benchmarks to arrow/util/machine-benchmark.ccARROW-4235 - [GLib] Use "column_builder" in GArrowRecordBatchBuilderARROW-4236 - [JAVA] Distinct plasma client create exceptionsARROW-4245 - [Rust] Add Rustdoc header to each source fileARROW-4247 - [Packaging] Update verify script for 0.12.0ARROW-4251 - [C++] Add option to use vendored Boost in verify-release-candidate.shARROW-4262 - [Website] Blog post to give preview into using R and Arrow with Apache SparkARROW-4263 - [Rust] Donate DataFusionARROW-4265 - [C++] Automatic conversion between Table and std::vector<std::tuple<..>>ARROW-4268 - [C++] Add C primitive to Arrow:Type compile time in TypeTraitsARROW-4271 - [Rust] Move Parquet specific info to Parquet ReadmeARROW-4273 - [Release] Fix verification script to use cf201901ARROW-4277 - [C++] Add gmock to toolchainARROW-4281 - [CI] Use Ubuntu Xenial (16.04) VMs on Travis-CIARROW-4285 - [Python] Use proper builder interface for serializationARROW-4287 - [C++] Ensure minimal bison version on OSX for ThriftARROW-4289 - [C++] Forward AR and RANLIB to thirdparty buildsARROW-4290 - [C++/Gandiva] Support detecting correct LLVM version in HomebrewARROW-4291 - [Dev] Support selecting features in release scriptsARROW-4294 - [Plasma] Add support for evicting objects to external storeARROW-4297 - [C++] Fix build for 32-bit MSYS2ARROW-4298 - [Java] Building Flight fails with OpenJDK 11ARROW-4299 - [Ruby] Depend on the same version as Red ArrowARROW-4300 - [C++] Restore apache-arrow Homebrew recipe and define process for maintaining and updating for releasesARROW-4303 - [Gandiva/Python] Build LLVM with RTTI in manylinux1 containerARROW-4305 - [Rust] Fix parquet version number in READMEARROW-4307 - [C++] FIx doxygen warnings, include doxygen warning checks in CI lintingARROW-4310 - [Website] Update install document for 0.12.0ARROW-4313 - Define general benchmark database schemaARROW-4315 - [Website] Home page of https://arrow.apache.org/ does not mention Go or RustARROW-4318 - [C++] Add Tensor::CountNonZeroARROW-4321 - [CI] Setup conda-forge channel globally in docker containersARROW-4330 - [C++] Use FindThreads.cmake to handle -pthread compiler/link optionsARROW-4331 - [C++] Extend Scalar Datum to support more typesARROW-4332 - [Website] Instructions and scripts for publishing web site appear to be incorrectARROW-4334 - [CI] Setup conda-forge channel globally in travis buildsARROW-4335 - [C++] Better document sparse tensor supportARROW-4336 - [C++] Default BUILD_WARNING_LEVEL to CHECKINARROW-4339 - [C++] rewrite cpp/README shorter, with a separate contribution guideARROW-4340 - [C++] Update IWYU version in the `lint` dockerfileARROW-4341 - [C++] Use TypedBufferBuilder in BooleanBuilder
ARROW-4344 - [Java] Further cleanup maven outputARROW-4345 - [C++] Add Apache 2.0 license file to the Parquet-testing repositoryARROW-4346 - [C++] Fix compiler warnings with gcc 8.2.0ARROW-4352 - [C++] Add support for system Google TestARROW-4353 - [CI] Add jobs for 32-bit and 64-bit MinGWARROW-4358 - [Gandiva][Crossbow] Trusty build brokenARROW-4361 - [Website] Update commiters listARROW-4362 - [Java] Test OpenJDK 11 in CIARROW-4363 - [C++] Add CMake format checksARROW-4372 - [C++] Embed precompiled bitcode in the gandiva libraryARROW-4373 - [Packaging] Travis fails to deploy conda packages on OSXARROW-4375 - [CI] Sphinx dependencies were removed from docs conda environmentARROW-4376 - [Rust] Implement from_buf_reader for csv::ReaderARROW-4377 - [Rust] Implement std::fmt::Debug for all PrimitiveArraysARROW-4379 - Register pyarrow serializers for collections.Counter and collections.deque.ARROW-4383 - [C++] Use the CMake's standard find featuresARROW-4386 - [Rust] Implement Date and Time ArraysARROW-4388 - [Go] add DimNames() method to tensor Interface?ARROW-4393 - [Rust] coding style: apply 90 characters per line limitARROW-4396 - Update Typedoc to support TypeScript 3.2ARROW-4397 - [C++] dim_names in Tensor and SparseTensorARROW-4399 - [C++] Remove usage of "extern template class" from NumericArray
ARROW-4401 - [Python] Alpine dockerfile fails to build because pandas requires numpy as build dependencyARROW-4406 - Ignore "*_$folder$" files on S3ARROW-4408 - [CPP/Doc] Remove outdated Parquet documentationARROW-4422 - [Plasma] Enforce memory limit in plasma, rather than relying on dlmalloc_set_footprint_limitARROW-4423 - [C++] Update version of vendored gtest to 1.8.1ARROW-4424 - [Python] Manylinux CI builds failingARROW-4425 - Add link to 'Contributing' page in the top-level Arrow READMEARROW-4430 - [C++] add unit test for currently unused append methodARROW-4431 - [C++] Build gRPC as ExternalProject without allowing it to build its vendored dependenciesARROW-4435 - [C#] Add .sln file and minor .csproj fix upsARROW-4436 - [Documentation] Clarify instructions for building documentationARROW-4442 - [JS] Overly broad type annotation for Chunked typeId leading to type mismatches in generated typingARROW-4444 - [Testing] Add DataFusion test files to arrow-testing repoARROW-4445 - [C++][Gandiva] Run Gandiva-LLVM tests in AppveyorARROW-4446 - [Python] Run Gandiva tests on Windows and AppveyorARROW-4448 - [JAVA][Flight] Flaky Flight java testARROW-4449 - [Rust] Convert File to T: Read + Seek for schema inferenceARROW-4454 - [C++] fix unused parameter warningsARROW-4455 - [Plasma] g++ 8 reports class-memaccess warningsARROW-4459 - [Testing] Add git submodule for arrow-testing data filesARROW-4460 - [Website] Write blog post to announce DataFusion donationARROW-4461 - [C++] Expose bit-util methods for binary boolean operations that don't allocateARROW-4462 - [C++] Upgrade LZ4 v1.7.5 to v1.8.3 to compile with VS2017ARROW-4464 - [Rust] [DataFusion] Add support for LIMITARROW-4466 - [Rust] [DataFusion] Add support for Parquet data sourcesARROW-4468 - [Rust] Implement BitAnd/BitOr for &Buffer (with SIMD)ARROW-4472 - [Website][Python] Blog post about Python string memory use improvements in 0.12ARROW-4475 - [Python] Serializing objects that contain themselvesARROW-4476 - [Rust] [DataFusion] Post donation clean up tasksARROW-4481 - [Website] Instructions for publishing web site are missing a stepARROW-4483 - [Website] Fix broken link (author) in DataFusion blog postARROW-4485 - [CI] Determine maintenance approach to pinned conda-forge binutils packageARROW-4486 - [Python][CUDA] pyarrow.cuda.Context.foreign_buffer should have a `base=None` argumentARROW-4488 - [Rust] From AsRef<[u8]> for Buffer does not ensure correct paddingARROW-4489 - [Rust] PrimitiveArray.value_slice performs bounds checking when it should notARROW-4490 - [Rust] Add explicit SIMD vectorization for boolean ops in "array_ops"ARROW-4491 - [Python] Remove usage of std::to_string and std::stoiARROW-4499 - [Python][CI] Upgrade to latest flake8 3.7.5 in travis_lint.shARROW-4502 - [C#] Add support for zero-copy readsARROW-4506 - [Ruby] Add Arrow::RecordBatch#raw_recordsARROW-4513 - [Rust] Implement BitAnd/BitOr for &BitmapARROW-4517 - [JS] remove version number as it is not usedARROW-4518 - [JS] add jsdelivr to package.jsonARROW-4528 - [C++] Update lint docker container to LLVM-7ARROW-4529 - [C++] Add test coverage for BitUtils::RoundDownARROW-4531 - [C++] Handling of non-aligned slices in Sum kernelARROW-4537 - [CI] Suppress shell warning on travis-ciARROW-4539 - [Java]List vector child value count not set correctlyARROW-4540 - [Rust] Add basic JSON readerARROW-4543 - [C#] Update Flat Buffers code to latest versionARROW-4546 - [C++] LICENSE.txt should be updated.ARROW-4547 - [Python][Documentation] Update python/development.rst with instructions for CUDA-enabled buildsARROW-4556 - [Rust] Preserve order of JSON inferred schemaARROW-4558 - [C++][Flight] Avoid undefined behavior with gRPC memory optimizationsARROW-4560 - [R] array() needs to take single input, not ...ARROW-4562 - [C++][Flight] Create outgoing composite grpc::ByteBuffer instead of allocating contiguous slice and copying IpcPayload into itARROW-4564 - [C++] IWYU docker image silently failsARROW-4565 - [R] Reading records with all non-null decimals SEGFAULTsARROW-4568 - [C++] Add version macros to headersARROW-4572 - [C++] Remove memory zeroing from PrimitiveAllocatingUnaryKernelARROW-4583 - [Plasma] There are bugs reported by code scan toolARROW-4586 - [Rust] Remove arrow/mod.rs as it is not neededARROW-4589 - [Rust] [DataFusion] Implement projection push down query optimizer ruleARROW-4590 - [Rust] Add explicit SIMD vectorization for comparison ops in "array_ops"ARROW-4592 - [GLib] Stop configure immediately when GLib isn't availableARROW-4593 - [Ruby] Arrow::Array#[out_of_range] returns nilARROW-4594 - [Ruby] Arrow::StructArray#[] returns Arrow::Struct instead of Arrow::ArrayARROW-4595 - [Rust] [DataFusion] Implement DataFrame style APIARROW-4598 - [CI] Remove needless LLVM_DIR for macOSARROW-4599 - [C++] Add support for system GFlagsARROW-4602 - [Rust][ [DataFusion] Integrate query optimizer with ExecutionContextARROW-4603 - [Rust] [DataFusion] Execution context should allow in-memory data sources to be registeredARROW-4604 - [Rust] [DataFusion] Add benchmarks for SQL query executionARROW-4605 - [Rust] Move filter and limit code from DataFusion into compute moduleARROW-4609 - [C++] Use google benchmark from toolchainARROW-4610 - [Plasma] Avoid JNI from crashingARROW-4611 - [C++] Rework CMake third-party logicARROW-4612 - [Python] Use cython from PyPI for windows wheels buildARROW-4613 - [C++] Alpine build failing as libgtestd.so is not foundARROW-4614 - [C++/CI] Activate flight build in ci/docker_build_cpp.shARROW-4615 - [C++] Add checked_pointer_castARROW-4616 - [C++] Log message in BuildUtils as STATUSARROW-4618 - [Docker] Makefile to build dependent docker imagesARROW-4619 - [R]: Fix the autobrew scriptARROW-4620 - [C#] Add unit tests for "Types" in arrow/csharpARROW-4623 - [R] update Rcpp dependencyARROW-4628 - [Rust] [DataFusion] Implement type coercion query optimizer ruleARROW-4632 - [Ruby] Add BigDecimal#to_arrowARROW-4634 - [Rust] [Parquet] Reorganize test_common mod to allow more test util codes.ARROW-4637 - [Python] Avoid importing Pandas unless necessaryARROW-4638 - [R] install instructions using brewARROW-4640 - [Python] Add docker-compose configuration to build and test the project without pandas installedARROW-4643 - [C++] Add compiler diagnostic color when using NinjaARROW-4644 - [C++/Docker] Build Gandiva in the docker containersARROW-4645 - [C++/Packaging] Ship Gandiva with OSX and Windows wheelsARROW-4646 - [C++/Packaging] Ship gandiva with the conda-forge packagesARROW-4655 - [Packaging] Parallelize binary uploadARROW-4662 - [Python] Add type_codes property in UnionTypeARROW-4667 - [C++] Suppress unused function warnings with MinGWARROW-4670 - [Rust] compute::sum performance issueARROW-4671 - [C++] MakeBuilder doesn't support Type::DICTIONARYARROW-4673 - [C++] Implement AssertDatumEqualsARROW-4676 - [C++] Add support for debug build with MinGWARROW-4678 - [Rust] Minimize unstable feature usageARROW-4679 - [Rust] [DataFusion] Implement in-memory DataSourceARROW-4681 - [Rust] [DataFusion] Implement parallel query execution using threadsARROW-4686 - Only accept 'y' or 'n' in merge_arrow_pr.py promptsARROW-4689 - [Go] add support for WASMARROW-4690 - [Python] Building TensorFlow compatible wheels for ArrowARROW-4692 - [Format][Documentation] Add more details about "sidecar" to flight protoARROW-4693 - [CI] Build boost library with multi precisionARROW-4697 - [C++] Add URI parsing facilityARROW-4703 - [C++] Upgrade dependency versionsARROW-4705 - [Rust] CSV reader should show line number and error message when failing to parse a lineARROW-4707 - [C++] move BitsetStack to bit-util.hARROW-4718 - Add ArrowStreamWriter/Reader ctors that leave open the underlying StreamARROW-4727 - [Rust] Implement ability to check if two schemas are the sameARROW-4730 - [C++] Add docker-compose entry for testing Fedora build with system packagesARROW-4731 - [C++] Add docker-compose entry for testing Ubuntu Xenial build with system packagesARROW-4732 - [C++] Add docker-compose entry for testing Debian Testing build with system packagesARROW-4733 - [C++] Add CI entry that builds without the conda-forge toolchain but with system packagesARROW-4734 - [Go] Add option to write a header for CSV writerARROW-4735 - [Go] Benchmark strconv.Format vs. fmt.Sprintf for CSV writerARROW-4739 - [Rust] [DataFusion] It should be possible to share a logical plan between threadsARROW-4740 - [Java] Upgrade to JUnit 5ARROW-4743 - [Java] Fix documentation in arrow memory moduleARROW-4745 - [C++][Documentation] Document process for replicating static_crt builds on windowsARROW-4749 - [Rust] RecordBatch::new() should return result instead of panickingARROW-4751 - [C++] Add pkg-config to conda_env_cpp.ymlARROW-4754 - [CI][Java] Flaky TestAuth Flight testARROW-4756 - [CI] document the procedure to update docker image for manylinux1 buildsARROW-4758 - [Flight] Build fails on Mac due to missing Schema_generated.hARROW-4769 - [Rust] Improve array limit function where max records > lenARROW-4772 - Provide new ORC adapter interface that allow user to specify row numberARROW-4776 - [C++] DictionaryBuilder should support bootstrapping from an existing dict typeARROW-4777 - [C++/Python] manylinux1: Update lz4 to 1.8.3ARROW-4778 - [C++/Python] manylinux1: Update Thrift to 0.12.0ARROW-4782 - [C++] Prototype scalar and array expression types for developing deferred operator algebraARROW-4786 - [C++/Python] Support better parallelisation in manylinux1 base buildARROW-4789 - [C++] Deprecate and and later remove arrow::io::ReadableFileInterfaceARROW-4790 - [Python/Packaging] Update manylinux docker image in crossbow taskARROW-4791 - Unused dependencies in arrow and datafusionARROW-4794 - [Python] Make pandas an optional test dependencyARROW-4797 - [Plasma] Avoid store crash if not enough memory is availableARROW-4801 - [GLib] Suppress pkgconfig.generate() warningsARROW-4808 - [Java][Vector] Convenience methods for setting decimal vectorARROW-4812 - [Rust] [DataFusion] Table.scan() should return one iterator per partitionARROW-4817 - [Rust] [DataFusion] Small re-org of modulesARROW-4818 - [Rust] [DataFusion] Parquet data source does not support null valuesARROW-4826 - [Go] export Flush method for CSV writerARROW-4831 - [C++] CMAKE_AR is not passed to ZSTD thirdparty dependencyARROW-4833 - [Release] Document how to update the brew formula in the release management guideARROW-4834 - [R] Feature flag to disable parquetARROW-4835 - [GLib] Add boolean operationsARROW-4837 - [C++] Support c++filt on a custom path in the run-test.sh scriptARROW-4839 - [C#] Add NuGet supportARROW-4843 - [Rust] [DataFusion] Parquet data source should support DATEARROW-4846 - [Java] Update Jackson to 2.9.8ARROW-4849 - [C++] Add docker-compose entry for testing Ubuntu Bionic build with system packagesARROW-4854 - [Rust] Use Array Slice for limit kernelARROW-4855 - [Packaging] Generate default package version based on cpp tags in crossbow.pyARROW-4858 - [Flight][Python] Enable custom FlightDataStream in PythonARROW-4859 - [GLib] Add garrow_numeric_array_mean()ARROW-4862 - [GLib] Add GArrowCastOptions::allow-invalid-utf8 propertyARROW-4862 - [GLib] Add GArrowCastOptions::allow-invalid-utf8 propertyARROW-4865 - [Rust] Support casting lists and primitives to listsARROW-4873 - [C++] Clarify documentation about how to use external ARROW_PACKAGE_PREFIX while also using CONDA dependency resolutionARROW-4878 - [C++] ARROW_DEPENDENCY_SOURCE=CONDA does not work properly with MSVCARROW-4882 - [GLib] Add "Sum" functionsARROW-4887 - [GLib] Add garrow_array_count()ARROW-4889 - [C++] Add STATUS messages for Protobuf in CMakeARROW-4891 - [C++] ZLIB include directories not addedARROW-4892 - [Rust] [DataFusion] Move SQL parser and planner into sql packageARROW-4893 - [C++] conda packages should use $PREFIX inside of conda-buildARROW-4894 - [Rust] [DataFusion] Remove all uses of panic! from aggregate.rsARROW-4895 - [Rust] [DataFusion] Move error.rs to top level packageARROW-4896 - [Rust] [DataFusion] Remove all uses of panic! from testsARROW-4897 - [Rust] [DataFusion] Improve RustdocARROW-4898 - [C++] Old versions of FindProtobuf.cmake use ALL-CAPS for variablesARROW-4899 - [Rust] [DataFusion] Remove all uses of panic! from expression.rsARROW-4901 - [Go] Run tests in AppveyorARROW-4905 - [C++][Plasma] Remove dlmalloc from client libraryARROW-4907 - [CI] Add docker container to inspect docker contextARROW-4908 - [Rust] [DataFusion] Add support for parquet date/time in int32/64 encodingARROW-4909 - [CI] Use hadolint to lint DockerfilesARROW-4910 - [Rust] [DataFusion] Remove all uses of unimplemented!ARROW-4915 - [GLib] Add support for arrow::NullBuilderARROW-4922 - [Packaging] Use system libraris for .deb and .rpmARROW-4924 - [Ruby] Add Decimal128#to_s(scale=nil)ARROW-4925 - [Rust] [DataFusion] Remove duplicate implementations of collect_exprARROW-4926 - [Rust] [DataFusion] Update README for 0.13.0 releaseARROW-4929 - [GLib] Add garrow_array_count_values()ARROW-4932 - [GLib] Use G_DECLARE_DERIVABLE_TYPE macroARROW-4933 - [R] Autodetect Parquet support using pkg-configARROW-4937 - [R] Clean pkg-config related logicARROW-4939 - [Python] Add wrapper for "sum" kernelARROW-4940 - [Rust] Enhance documentation for datafusionARROW-4944 - [C++] Raise minimal required thrift-cpp to 0.11 in conda environmentARROW-4946 - [C++] Support detection of flatbuffers without FlatbuffersConfig.cmakeARROW-4947 - [Flight][C++/Python] Remove redundant schema parameter in DoGetARROW-4951 - [C++] Turn off cpp benchmarks in cpp docker imagesARROW-4955 - [GLib] Add garrow_file_is_closed()ARROW-4964 - [Ruby] Add closed check if available on auto closeARROW-4969 - [C++] Set RPATH in correct order for test executables on OSXARROW-4977 - [Ruby] Add support for building on WindowsARROW-4978 - [Ruby] Fix wrong internal variable name for table dataARROW-4979 - [GLib] Add missing lock to garrow::GIOInputStreamARROW-4980 - [GLib] Use GInputStream as the parent of GArrowInputStreamARROW-4981 - [Ruby] Add support for CSV data encoding conversionARROW-4983 - [Plasma] Unmap memory when the client is destroyedARROW-4994 - [website] Update Details for ptgoetzARROW-4995 - [R] Make sure winbuilder tests pass for packageARROW-4996 - [Plasma] There are many log files in /tmpARROW-5003 - [R] remove dependency on withrARROW-5006 - [R] parquet.cpp does not include enough RcppARROW-5011 - [Release] Add support in the source release script for custom hashARROW-5013 - [Rust] [DataFusion] Refactor runtime expression supportARROW-5014 - [Java] Fix typos in Flight moduleARROW-5018 - [Release] Include JavaScript implementationARROW-5032 - [C++] Headers in vendored/datetime directory aren't installedARROW-5041 - [Release][C++] use bundled gtest and gmock in verify-release-candidate.batARROW-5075 - [Release] Add 0.13.0 release noteARROW-5084 - [Website] Blog post / release announcement for 0.13.0PARQUET-1477 - Thrift crypto updatesPARQUET-1508 - [C++] Enable reading from ByteArray and FixedLenByteArray decoders directly into arrow::BinaryBuilder or arrow::BinaryDictionaryBuilderPARQUET-1519 - [C++] Remove use of "extern template class" from parquet/column_reader.hPARQUET-1521 - [C++] Do not use "extern template class" with parquet::ColumnWriterPARQUET-1525 - [C++] remove dependency on getopt in parquet toolsApache Arrow 0.12.1 (2019-02-25)
Bug Fixes
ARROW-3564 - [Python] writing version 2.0 parquet format with dictionary encoding enabledARROW-4255 - [C++] Schema::GetFieldIndex is not thread-safeARROW-4267 - [Python/C++][Parquet] Segfault when reading rowgroups with duplicated columnsARROW-4323 - [Packaging] Fix failing OSX clang conda forge buildsARROW-4367 - [C++] StringDictionaryBuilder segfaults on Finish with only null entriesARROW-4374 - [C++] DictionaryBuilder does not correctly report length and null_countARROW-4492 - [Python] Failure reading Parquet column as pandas Categorical in 0.12ARROW-4501 - [C++] Unique returns non-unique stringsARROW-4582 - [C++/Python] Memory corruption on Pandas->Arrow conversionARROW-4629 - [Python] Pandas to arrow conversion slowed down by local importsARROW-4636 - [Python/Packaging] Crossbow builds for conda-osx fail on upload with Ruby linkage errorsARROW-4647 - [Packaging] dev/release/00-prepare.sh fails for minor version changes
New Features and Improvements
ARROW-4291 - [Dev] Support selecting features in release scriptsARROW-4298 - [Java] Building Flight fails with OpenJDK 11ARROW-4373 - [Packaging] Travis fails to deploy conda packages on OSX
Apache Arrow 0.12.0 (2019-01-20)
New Features and Improvements
ARROW-45 - [Python] Add unnest/flatten function for List typesARROW-536 - [C++] Provide non-SSE4 versions of functions that use CPU intrinsics for older processorsARROW-554 - [C++] Implement functions to conform unequal dictionaries amongst multiple Arrow arraysARROW-766 - [C++] Introduce zero-copy "StringPiece" typeARROW-854 - [Format] Support sparse tensorARROW-912 - [Python] Account for multiarch systems in development.rstARROW-1019 - [C++] Implement input stream and output stream with Gzip codecARROW-1055 - [C++] GPU support library developmentARROW-1262 - [Packaging] Packaging automation in arrow-distARROW-1423 - [C++] Create non-owned CudaContext from context handle provided by thirdparty userARROW-1492 - [C++] Type casting function kernel suiteARROW-1688 - [Java] Fail build on checkstyle warningsARROW-1696 - [C++] Add codec benchmarksARROW-1822 - [C++] Add SSE4.2-accelerated hash kernels and use if host CPU supportsARROW-1993 - [Python] Add function for determining implied Arrow schema from pandas.DataFrameARROW-1994 - [Python] Test against Pandas masterARROW-2183 - [C++] Add helper CMake function for globbing the right header filesARROW-2211 - [C++] Use simpler hash functions for integersARROW-2216 - [CI] CI descriptions and envars are misleadingARROW-2337 - [Scripts] Windows release verification script should use boost DSOs instead of static linkageARROW-2374 - [Rust] Add support for array of List
ARROW-2475 - [Format] Confusing array length descriptionARROW-2476 - [Python/Question] Maximum length of an Array created from ndarrayARROW-2483 - [Rust] use bit-packing for boolean vectorsARROW-2504 - [Website] Add ApacheCon NA linkARROW-2535 - [Python] Provide pre-commit hooks that check flake8ARROW-2560 - [Rust] The Rust README should include Rust-specific information on contributingARROW-2624 - [Python] Random schema and data generator for Arrow conversion and Parquet testingARROW-2637 - [C++/Python] Build support and instructions for development on Alpine LinuxARROW-2648 - [Packaging] Follow up packaging tasksARROW-2653 - [C++] Refactor hash table supportARROW-2670 - [C++/Python] Add Ubuntu 18.04 / gcc7 as a nightly buildARROW-2673 - [Python] Add documentation + docstring for ARROW-2661ARROW-2684 - [Python] Various documentation improvementsARROW-2712 - [C#] Initial C# .NET libraryARROW-2720 - [C++] Clean up cmake CXX_STANDARD and PIC flag settingARROW-2759 - Export notification socket of PlasmaARROW-2803 - [C++] Put hashing function into src/arrow/utilARROW-2807 - [Python] Enable memory-mapping to be toggled in get_reader when reading Parquet filesARROW-2808 - [Python] Add unit tests for ProxyMemoryPool, enable new default MemoryPool to be constructedARROW-2919 - [C++] Improve error message when listing empty HDFS fileARROW-2968 - [R] Multi-threaded conversion from Arrow table to R data.frameARROW-2995 - [CI] Build Python libraries in same run when running C++ unit tests so project does not need to be rebuilt again right awayARROW-3020 - [Python] Addition of option to allow empty Parquet row groupsARROW-3038 - [Go] add support for StringArrayARROW-3063 - [Go] move list of supported/TODO features to confluenceARROW-3070 - [Release] Host binary artifacts for RCs and releases on ASF Bintray account instead of dist/mirror systemARROW-3108 - [C++] arrow::PrettyPrint for Table instancesARROW-3126 - [Python] Make Buffered* IO classes available to Python, incorporate into input_stream, output_stream factory functionsARROW-3131 - [Go] add test for Go-1.11ARROW-3161 - [Packaging] Ensure to run pyarrow unit tests in conda and wheel buildsARROW-3169 - [C++] Break array-test.cc and array.cc into multiple compilation unitsARROW-3184 - [C++] Add modular build targets, "all" target, and require explicit target when invoking make or ninjaARROW-3194 - [Java] Fix setValueCount in spitAndTransfer for variable width vectorsARROW-3199 - [Plasma] Check for EAGAIN in recvmsg and sendmsgARROW-3209 - [C++] Rename libarrow_gpu to libarrow_cudaARROW-3230 - [Python] Missing comparisons on ChunkedArray, TableARROW-3233 - [Python] Sphinx documentation for pyarrow.cuda GPU supportARROW-3248 - [C++] Arrow tests should have label "arrow"ARROW-3254 - [C++] Add option to ADD_ARROW_TEST to compose a test executable from multiple .cc files containing unit testsARROW-3260 - [CI] Make linting a separate jobARROW-3272 - [Java] Document checkstyle deviations from Google style guideARROW-3273 - [Java] checkstyle - fix javadoc styleARROW-3278 - [Python] Retrieve StructType's and StructArray's field by nameARROW-3291 - [C++] Convenience API for constructing arrow::io::BufferReader from std::stringARROW-3293 - [C++] Test Flight RPC in Travis CIARROW-3296 - [Python] Add Flight support to manylinux1 wheelsARROW-3303 - [C++] Enable example arrays to be written with a simplified JSON representationARROW-3306 - [R] Objects and support functions different kinds of arrow::BufferARROW-3307 - [R] Convert chunked arrow::Column to R vectorARROW-3310 - [R] Create wrapper classes for various Arrow IO interfacesARROW-3312 - [R] Use same .clang-format file for both R binding C++ code and main C++ codebaseARROW-3315 - [R] Support for multi-threaded conversions from RecordBatch, Table to R data.frameARROW-3318 - [C++] Convenience method for reading all batches from an IPC stream or file as arrow::TableARROW-3323 - [Java] checkstyle - fix namingARROW-3331 - [C++] Add re2 to ThirdpartyToolchainARROW-3340 - [R] support for dates and time classesARROW-3347 - [Rust] Implement PrimitiveArrayBuilderARROW-3353 - [Packaging] Build python 3.7 wheelsARROW-3355 - [R] Support for factorsARROW-3358 - [Gandiva][C++] Replace usages of gandiva/status.h with arrow/status.hARROW-3362 - [R] Guard against null buffersARROW-3366 - [R] Dockerfile for docker-compose setupARROW-3368 - [Integration/CI/Python] Add dask integration test to docker-compose setupARROW-3380 - [Python] Support reading CSV files and more from a gzipped fileARROW-3381 - [C++] Implement InputStream for bz2 filesARROW-3383 - [Java] Run Gandiva tests in Travis CIARROW-3384 - [Gandiva] Sync remaining commits from gandiva repoARROW-3385 - [Java] [Gandiva] Deploy gandiva snapshot jars automaticallyARROW-3387 - [C++] Function to cast binary to string/utf8 with UTF8 validationARROW-3398 - [Rust] Update existing Builder to use MutableBuffer internallyARROW-3402 - [Gandiva][C++] Utilize common bitmap operation implementations in precompiled IR routinesARROW-3407 - [C++] Add UTF8 conversion modes in CSV reader conversion optionsARROW-3409 - [C++] Add streaming compression interfacesARROW-3421 - [C++] Add include-what-you-use setup to primary docker-compose.ymlARROW-3427 - [C++] Add Windows support, Unix static libs for double-conversion package in conda-forgeARROW-3429 - [Packaging] Add a script to release binaries that use source archive at dist.apache.orgtable bitARROW-3430 - [Packaging] Add workaround to verify 0.11.0ARROW-3431 - [GLib] Include Gemfile to archiveARROW-3432 - [Packaging] Variables aren't expanded Subversion commit messageARROW-3433 - [C++] Validate re2 with Windows toolchain, EPARROW-3439 - [R] R language bindings for Feather formatARROW-3440 - [Gandiva][C++] Remove outdated cpp/src/gandiva/README.md, add build documentation to cpp/README.mdARROW-3441 - [Gandiva][C++] Produce fewer test executablesARROW-3442 - [C++] Use dynamic linking for unit tests, ensure coverage working properly with clangARROW-3450 - [R] Wrap MemoryMappedFile classARROW-3451 - [Python] Allocate CUDA memory from a CUcontext created by numba.cudaARROW-3455 - [Gandiva][C++] Support pkg-config for GandivaARROW-3456 - [CI] Reuse docker images and optimize docker-compose containersARROW-3460 - [Packaging] Add a script to rebase master on local release branchARROW-3461 - [Packaging] Add a script to upload RC artifacts as the official releaseARROW-3462 - [Packaging] Update CHANGELOG for 0.11.0ARROW-3463 - [Website] Update for 0.11.0ARROW-3464 - [Packaging] Build shared libraries for gandiva fat JAR via crossbowARROW-3465 - [Documentation] Fix gen_apidocs' docker imageARROW-3469 - [Gandiva] add travis entry for gandiva on OSXARROW-3472 - [Gandiva] remove gandiva helpers libraryARROW-3473 - [Format] Update Layout.md document to clarify use of 64-bit array lengthsARROW-3474 - [GLib] Extend gparquet API with get_schema and read_columnARROW-3479 - [R] Support to write record_batch as streamARROW-3482 - [C++] Build with JEMALLOC by defaultARROW-3487 - [Gandiva] simplify NULL_IF_NULL functions that can return errorsARROW-3488 - [Packaging] Separate crossbow task definition files for packaging and testsARROW-3489 - [Gandiva] Support for in expressionsARROW-3490 - [R] streaming arrow objects to output streamsARROW-3492 - [C++] Build jemalloc in parallelARROW-3493 - [Java] Document BOUNDS_CHECKING_ENABLEDARROW-3499 - [R] Expose arrow::ipc::Message typeARROW-3501 - [Gandiva] Enable building with gcc 4.8.x on Ubuntu Trusty, similar distrosARROW-3504 - [Plasma] Add support for Plasma Client to put/get raw bytes without pyarrow serialization.ARROW-3505 - [R] Read record batch and tableARROW-3506 - [Packaging] Nightly tests for docker-compose imagesARROW-3508 - [C++] Build against double-conversion from conda-forgeARROW-3515 - Introduce NumericTensor classARROW-3518 - [C++] Detect HOMEBREW_PREFIX automaticallyARROW-3519 - [Gandiva] Add support for functions that can return variable len outputARROW-3521 - [GLib] Run Python using find_program in meson.buildARROW-3529 - [Ruby] Import Red ParquetARROW-3530 - [Java/Python] Add conversion for pyarrow.Schema from org.apache…pojo.SchemaARROW-3533 - [Python/Documentation] Use sphinx_rtd_theme instead of BootstrapARROW-3536 - [C++] Fast UTF8 validation functionsARROW-3537 - [Rust] Implement Tensor TypeARROW-3539 - [CI/Packaging] Update scripts to build against vendored jemallocARROW-3540 - [Rust] Incorporate BooleanArray into PrimitiveArrayARROW-3542 - [C++] Use unsafe appends when building array from CSVARROW-3545 - [C++/Python] Normalize child/field terminology with StructTypeARROW-3547 - [R] Protect against Null crash when reading from RecordBatchARROW-3548 - Speed up storing small objects in the object store.ARROW-3551 - Change MapD to OmniSci on Powered By pageARROW-3553 - [R] Error when losing data on int64, uint64 conversions to doubleARROW-3555 - [Plasma] Unify plasma client get function using metadata.ARROW-3556 - [CI] Disable optimizations on WindowsARROW-3557 - [Python] Set language_level in Cython sourcesARROW-3558 - [Plasma] Remove fatal error when plasma client calls get on an unsealed object that it created.ARROW-3559 - Statically link libraries for plasma_store_server executable.ARROW-3562 - [R] Disallow creation of objects with null shared_ptr
ARROW-3563 - [C++] Declare public link dependencies so arrow_static, plasma_static automatically pull in transitive dependenciesARROW-3566 - Clarify that the type of dictionary encoded field should be the encoded(index) typeARROW-3567 - [Gandiva] [GLib] Add GLib bindings of GandivaARROW-3568 - [Packaging] Run pyarrow unittests for windows wheelsARROW-3569 - [Packaging] Run pyarrow unittests when building conda packageARROW-3574 - Fix remaining bug with plasma static versus shared libraries.ARROW-3575 - [Python] New documentation page for CSV readerARROW-3576 - [Python] Expose compressed file readers as NativeFileARROW-3577 - [Go] add support for ChunkedArrayARROW-3581 - [Gandiva][C++] ARROW_PROTOBUF_USE_SHARED isn't usedARROW-3582 - [CI] Gandiva C++ build is always triggeredARROW-3583 - [Python/Java] Create RecordBatch from VectorSchemaRootARROW-3584 - [Go] add support for TableARROW-3587 - [Python] Efficient serialization for Arrow Objects (array, table, tensor, etc)ARROW-3588 - [Java] checkstyle - fix licenseARROW-3589 - [Gandiva] Make it possible to compile gandiva without JNIARROW-3591 - [R] Support to collect decimal typeARROW-3592 - [Python] Get BinaryArray value as zero copy memory viewARROW-3597 - [Gandiva] gandiva should integrate with ADD_ARROW_TEST for testsARROW-3600 - [Packaging] Support Ubuntu 18.10ARROW-3601 - [Rust] Release 0.11.0ARROW-3602 - [Gandiva] [Python] Add preliminary Cython bindings for GandivaARROW-3603 - [Gandiva][C++] Can't build with vendored BoostARROW-3605 - Remove AE library from plasma header files.ARROW-3607 - [Java] delete() method via JNI for plasmaARROW-3608 - [R] Support for time32 and time64 array typesARROW-3609 - [Gandiva] Move benchmark tests out of unit testARROW-3610 - [C++] Add interface to turn stl_allocator into arrow::MemoryPoolARROW-3611 - Give error more quickly when pyarrow serialization context is used incorrectly.ARROW-3612 - [Go] implement RecordBatch and RecordBatchReaderARROW-3615 - [R] Support for NaNARROW-3616 - [Java] checkstyle - fix remaining coding checksARROW-3618 - [Packaging/Documentation] Add `-c conda-forge` option to avoid PackagesNotFoundErrorARROW-3620 - [Python] Document multithreading options in Sphinx and add to api.rstARROW-3621 - [Go] implement TableBatchReaderARROW-3622 - [Go] implement Schema.EqualARROW-3623 - [Go] implement Field.EqualARROW-3624 - [Python/C++] Support for zero-sized device buffersARROW-3625 - [Go] add examples for Table, Record and {Table,Record}ReaderARROW-3626 - [Go] add a CSV TableReaderARROW-3627 - [Go] add RecordBatchBuilderARROW-3629 - [Python] Add write_to_dataset to Python Sphinx API listingARROW-3630 - [Plasma] [GLib] Add GLib bindings of PlasmaARROW-3632 - [Packaging] Update deb names in dev/tasks/tasks.yml in dev/release/00-prepare.shARROW-3633 - [Packaging] Update deb names in dev/tasks/tasks.yml for 0.12.0ARROW-3636 - [C++/Python] Update arrow/python/pyarrow_api.hARROW-3638 - [C++][Python] Move reading from Feather as Table feature to C++ from PythonARROW-3639 - [Packaging] Run gandiva nightly packaging tasksARROW-3640 - [Go] add support for TensorsARROW-3641 - [C++/Python] remove public keyword from Cython api functionsARROW-3642 - [C++] Add arrowConfig.cmake generationARROW-3644 - [Rust] Implement ListArrayBuilderARROW-3645 - [Python] Document compression support in SphinxARROW-3646 - [Python] Add convenience factories to create IO streamsARROW-3647 - [R] Crash after unloading bit64 packageARROW-3648 - [Plasma] Add API to get metadata and data at the same timeARROW-3649 - [Rust] Refactor MutableBuffer's resizeARROW-3656 - [C++] Allow whitespace in numeric CSV fieldsARROW-3657 - [R] Require bit64 packageARROW-3659 - [C++] Clang Travis build (matrix entry 2) might not actually be using clangARROW-3660 - [C++] Don't unnecessarily lock MemoryMappedFile for resizing in readonly filesARROW-3661 - [Gandiva][GLib] Improve constant nameARROW-3662 - [C++] Add a const overload to MemoryMappedFile::GetSizeARROW-3664 - [Rust] Add benchmark for PrimitiveArrayBuilderARROW-3665 - [Rust] Implement StructArrayBuilderARROW-3666 - [C++] Improve CSV parser performanceARROW-3672 - [Go] implement Time32 arrayARROW-3673 - [Go] implement Time64 arrayARROW-3674 - [Go] implement Date32 arrayARROW-3675 - [Go] implement Date64 arrayARROW-3677 - [Go] implement FixedSizedBinary arrayARROW-3681 - [Go] add benchmarks for CSV readerARROW-3682 - [Go] unexport encoding/csv.Reader from CSV readerARROW-3683 - [Go] add functional-option style to CSV readerARROW-3684 - [Go] add chunk size option to CSV readerARROW-3692 - [Gandiva] [Ruby] Add Ruby bindings of GandivaARROW-3693 - [R] Invalid buffer for empty characters with null dataARROW-3694 - [Java] Avoid superfluous string creation when logging level is disabledARROW-3695 - [Gandiva] Use add_arrow_lib()ARROW-3696 - [C++] Add feather::TableWriter::Write(table)ARROW-3697 - [Ruby] Add schema#[]ARROW-3701 - [Gandiva] Add support for decimal operationsARROW-3708 - [Packaging] Nightly CentOS builds are failingARROW-3713 - [Rust] Implement BinaryArrayBuilderARROW-3718 - [Gandiva] Remove spurious gtest includeARROW-3719 - [GLib] Support read/write table to/from FeatherARROW-3720 - [GLib] Use "indices" instead of "indexes"ARROW-3721 - [Gandiva] [Python] Support all Gandiva literalsARROW-3722 - [C++] Allow specifying column types to CSV readerARROW-3723 - [Plasma] [Ruby] Add Ruby bindings of PlasmaARROW-3724 - [GLib] Update gitignoreARROW-3725 - [GLib] Add field readers to GArrowStructDataTypeARROW-3726 - [Rust] CSV Reader & WriterARROW-3727 - [Python] Document use of pyarrow.foreign_buffer, cuda.foreign_buffer in SphinxARROW-3731 - [R] R API for reading and writing Parquet filesARROW-3733 - [GLib] Add to_string() to GArrowTable and GArrowColumnARROW-3736 - [CI/Docker] Ninja test in docker-compose run cpp hangsARROW-3738 - [C++] Add CSV conversion option to parse ISO8601-like timestamp stringsARROW-3741 - [R] Add support for arrow::compute::Cast to convert Arrow arrays from one type to anotherARROW-3743 - [Ruby] Add support for saving/loading FeatherARROW-3744 - [Ruby] Use garrow_table_to_string() in Arrow::Table#to_sARROW-3746 - [Gandiva] [Python] Make it possible to list all functions registered with GandivaARROW-3747 - [C++] Flip order of data members in arrow::Decimal128ARROW-3748 - [GLib] Add GArrowCSVReaderARROW-3749 - [GLib] Typos in documentation and test case nameARROW-3751 - [Python] Add more cython bindings for gandivaARROW-3752 - [C++] Remove unused status::ArrowErrorARROW-3753 - [Gandiva] Remove debug printARROW-3755 - [GLib] Support for CompressedInputStream, CompressedOutputStreamARROW-3760 - [R] Support Arrow CSV readerARROW-3773 - [C++] Remove duplicated AssertArraysEqual code in parquet/arrow/arrow-reader-writer-test.ccARROW-3778 - [C++] Don't put implementations in test-util.hARROW-3781 - [C++] Configure buffer size in arrow::io::BufferedOutputStreamARROW-3782 - [C++] Implement BufferedReader for C++ARROW-3784 - [R] Array with type fails with x is not a vectorARROW-3785 - [C++] Use double-conversion conda package in CI toolchainARROW-3787 - Implement From for BinaryArray
ARROW-3788 - [Ruby] Add support for CSV parser writtin in C++ARROW-3795 - [R] Support for retrieving NAs from INT64 arraysARROW-3796 - [Rust] Add Example for PrimitiveArrayBuilderARROW-3798 - [GLib] Add support for column type CSV read optionsARROW-3800 - [C++] Vendor a string_view backportARROW-3803 - [C++/Python] Split C++ and Python unit test Travis CI jobs, run all C++ tests (including Gandiva) togetherARROW-3807 - [R] Missing Field APIARROW-3819 - [Packaging] Update conda variant files to conform with feedstock after compiler migrationARROW-3821 - [Format/Documentation]: Fix typos and grammar issues in Flight.proto commentsARROW-3823 - [R] + buffer.complexARROW-3825 - [Python] The Python README.md does not show how to run the unit test suiteARROW-3826 - [C++] Determine if using ccache caching in Travis CI actually improves build timesARROW-3830 - [GLib] Add GArrowCodecARROW-3834 - [Doc] Merge Python & C++ and move to top-levelARROW-3836 - [C++] Add PREFIX option to ADD_ARROW_BENCHMARKARROW-3839 - [Rust] Add ability to infer schema in CSV readerARROW-3841 - [C++] warning: catching polymorphic type by valueARROW-3842 - [R] RecordBatchStreamWriter apiARROW-3844 - [C++] Remove ARROW_USE_SSE and ARROW_SSE3ARROW-3845 - [Gandiva] [GLib] Add GGandivaNodeARROW-3847 - [GLib] Remove unnecessary “\”.ARROW-3849 - Leverage Armv8 crc32 extension instructions to accelerate the hash computation for Arm64.ARROW-3851 - [C++] "make check-format" is slowARROW-3852 - [C++] used uninitialized warningARROW-3853 - [C++] Implement string to timestamp castARROW-3854 - [GLib] Deprecate garrow_gio_{input,output}_stream_get_raw()ARROW-3855 - [Rust] Schema/Field/Datatype should implement serde traitsARROW-3856 - [Ruby] Support compressed CSV save/loadARROW-3858 - [GLib] Use {class_name}_get_instance_privateARROW-3859 - [Java] Fix ComplexWriter backward incompatible changeARROW-3860 - [Gandiva] [C++] Add option to use -static-libstdc++ when building libgandiva_jni.soARROW-3862 - [C++] Improve dependencies download scriptARROW-3863 - [GLib] Use travis_retry with brew bundle commandARROW-3864 - [GLib] Add support for allow-float-truncate cast optionARROW-3865 - [Packaging] Add double-conversion dependency to conda forge recipes and the windows wheel buildARROW-3867 - [Documentation] Uploading binary realase artifacts to BintrayARROW-3868 - [Rust] Build against nightly Rust in CIARROW-3870 - [C++] Add Peek to InputStream APIARROW-3871 - [R] Replace usages of C++ GetValuesSafely with new methods on ArrayDataARROW-3878 - [Rust] Improve primitive typesARROW-3880 - [Rust] PrimitiveArray should support simple math operations
ARROW-3881 - [Rust] PrimitiveArray should support comparison operators
ARROW-3883 - [Rust] Update Rust README to reflect new functionalityARROW-3884 - [Python] Add LLVM6 to manylinux1 base imageARROW-3885 - [Rust] Update version to 0.12.0 and update release instructions on wikiARROW-3886 - [C++] Additional test cases for ARROW-3831ARROW-3891 - [Java] Remove Long.bitCount with simple bitmap operationsARROW-3893 - [C++] Improve adaptive int builder performanceARROW-3895 - [Rust] CSV reader should return Result> not Option>
ARROW-3899 - [Python] Table.to_pandas converts Arrow date32[day] to pandas datetime64[ns]ARROW-3900 - [GLib] Add garrow_mutable_buffer_set_data()ARROW-3905 - [Ruby] Add StructDataType#[]ARROW-3906 - [C++] Break builder.cc into multiple compilation unitsARROW-3908 - [Rust] Update rust dockerfile to use nightly toolchainARROW-3910 - [Python] Set date_as_object to True in *.to_pandas as default after deduplicating logic implementedARROW-3911 - [Python] Deduplicate datetime.date objects in Table.to_pandas internalsARROW-3912 - [Plasma][GLib] Add support for creating and referring objectsARROW-3913 - [Gandiva] [GLib] Add GGandivaLiteralNodeARROW-3914 - [C++/Python/Packaging] Docker-compose setup for Alpine linuxARROW-3916 - [Python] Support caller-provided filesystem in `ParquetWriter` constructorARROW-3921 - [CI][GLib] Log Homebrew outputARROW-3922 - [C++] improve the performance of bitmap operationsARROW-3924 - [Packaging][Plasma] Add support for Plasma deb/rpm packagesARROW-3925 - [Python] Include autoconf in Linux/macOS dependencies in conda environmentARROW-3928 - [Python] Add option to deduplicate PyBytes / PyString / PyUnicode objects in Table.to_pandas conversion pathARROW-3929 - [Go] improve memory usage of CSV reader to improve runtime performancesARROW-3930 - [C++] Random test data generation is slowARROW-3932 - [Python/Documentation] Include Benchmarks.md in Sphinx docsARROW-3934 - [Gandiva] Don't compile precompiled tests if ARROW_GANDIVA_BUILD_TESTS=offARROW-3938 - [Packaging] Stop to refer java/pom.xml to get version informationARROW-3939 - [Rust] Remove macro definition for ListArrayBuilderARROW-3945 - [Website] Blog post about Gandiva code donationARROW-3946 - [GLib] Add support for unionARROW-3948 - [CI][GLib] Set timeout to HomebrewARROW-3950 - [Plasma] Don't force loading the TensorFlow op on importARROW-3952 - [Rust] Specify edition="2018" in Cargo.tomlARROW-3958 - [Plasma] Reduce number of IPCsARROW-3959 - [Rust] Time and Timestamp SupportARROW-3960 - [Rust] remove extern crate for Rust 2018ARROW-3963 - [Packaging/Docker] Nightly test for building sphinx documentationsARROW-3964 - [Go] More readable example for csv.ReaderARROW-3967 - [Gandiva] [C++] Make gandiva/node.h publicARROW-3970 - [Gandiva][C++] Remove unnecessary boost dependenciesARROW-3971 - [Python] Remove APIs deprecated in 0.11 and priorARROW-3974 - [C++] Combine field_builders_ and children_ members in array/builder.hARROW-3982 - [C++] Allow "binary" input in simple JSON formatARROW-3983 - [Gandiva][Crossbow] Use static boost while packagingARROW-3984 - [C++] Exit with error if user hits zstd ExternalProject pathARROW-3986 - [C++] Write prose documentationARROW-3986 - [C++] Write prose documentationARROW-3987 - [Java] Benchmark results for ARROW-1807ARROW-3988 - [C++] Do not build unit tests by default in build systemARROW-3993 - [JS] CI Jobs FailingARROW-3994 - [C++] Remove ARROW_GANDIVA_BUILD_TESTS optionARROW-3995 - [CI] Use understandable names in Travis MatrixARROW-3997 - [C++] [Doc] Clarify dictionary encoding integer signedness (and width?)ARROW-4002 - [C++][Gandiva] Remove CMake version checkARROW-4004 - [GLib] Replace GPU with CUDAARROW-4005 - [Plasma] [GLib] Add gplasma_client_disconnect()ARROW-4006 - Add CODE_OF_CONDUCT.mdARROW-4009 - [CI] Run Valgrind and C++ code coverage in different buldsARROW-4010 - [C++] Enable Travis CI scripts to only build and install only certain targetsARROW-4015 - [Plasma] remove legacy interfaces for plasma managerARROW-4017 - [C++] Check and update vendored librariesARROW-4026 - [C++] Use separate modular $COMPONENT-test targets for unit testsARROW-4028 - [Rust] Merge parquet-rs codebaseARROW-4029 - [C++] Define and document naming convention for internal / private header files not to be installedARROW-4030 - [CI] Use travis_terminate to halt builds when a step failsARROW-4035 - [Ruby] Support msys2 mingw dependenciesARROW-4037 - [Packaging] Remove workaround to verify 0.11.0ARROW-4038 - [Rust] Add array_ops methods for boolean AND, OR, NOTARROW-4039 - [Python] Update link to 'development.rst' page from Python README.mdARROW-4042 - [Rust] Inconsistent method naming between BinaryArray and PrimitiveArrayARROW-4043 - [Packaging/Docker] Python tests on alpine miss pytest dependencyARROW-4044 - [Packaging/Python] Add hypothesis test dependency to pyarrow conda recipeARROW-4045 - [Packaging/Python] Add hypothesis test dependency to wheel crossbow testsARROW-4048 - [GLib] Return ChunkedArray instead of Array in gparquet_arrow_file_reader_read_columnARROW-4051 - [Gandiva] [GLib] Add support for null literalARROW-4054 - [Python] Update gtest, flatbuffers and OpenSSL in manylinux1 base imageARROW-4060 - [Rust] Add Parquet/Arrow schema converterARROW-4069 - [Python] Add tests for casting from binary to utf8ARROW-4075 - [Rust] Reuse array builder after calling finish()ARROW-4079 - [C++] Add machine benchmarksARROW-4080 - [Rust] Improving lengthy build times in AppveyorARROW-4082 - [C++] CMake tweaks: allow RelWithDebInfo, improve FindClangToolsARROW-4084 - [C++] Simplify Status and stringstream boilerplateARROW-4085 - [GLib] Use "field" for struct data typeARROW-4087 - [C++] Make CSV nulls configurableARROW-4093 - [C++] Deprecated method suggests wrong methodARROW-4098 - [Python] Deprecate pyarrow.open_stream,open_file in favor of pa.ipc.open_stream/open_fileARROW-4100 - [Gandiva][C++] Fix regex to ignore "." characterARROW-4102 - [C++] FixedSizeBinary identity cast not implementedARROW-4103 - [Documentation] Add README to docs/ rootARROW-4105 - Add rust-toolchain to enforce user to use nightly toolchain for buildingARROW-4107 - [Python] Use ninja in pyarrow manylinux1 buildARROW-4112 - [Packaging][Gandiva] Add support for deb packagesARROW-4116 - [Python] Clarify in development.rst that virtualenv cannot be used with miniconda/AnacondaARROW-4122 - [C++] Initialize some uninitialized class membersARROW-4127 - [Documentation] Add Docker build instructionsARROW-4129 - [Python] Fix syntax problem in benchmark docsARROW-4132 - [GLib] Add more GArrowTable constructorsARROW-4141 - [Ruby] Add support for creating schema from raw Ruby objectsARROW-4148 - [CI/Python] Disable ORC on nightly Alpine buildsARROW-4150 - [C++] Do not return buffers containing nullptr from internal allocationsARROW-4151 - [Rust] Restructure project directoriesARROW-4152 - [GLib] Remove an example to show Torch integrationARROW-4153 - [GLib] Add builder_append_value() for consistencyARROW-4154 - [GLib] Add GArrowDecimal128DataTypeARROW-4155 - [Rust] Implement array_ops::sum() for PrimitiveArray
ARROW-4156 - [C++] xcodebuild failure for cmake generated projectARROW-4158 - [Dev] Allow maintainers to use a GitHub API token when merging pull requestsARROW-4160 - [Rust] Add README and executable files to parquetARROW-4161 - [GLib] Add GPlasmaClientOptionsARROW-4162 - [Ruby] Add support for creating data types from descriptionARROW-4166 - [Ruby] Add support for saving to and loading from bufferARROW-4167 - [Gandiva] switch to arrow/util/variantARROW-4168 - [GLib] Use property to keep GArrowDataType passed in garrow_field_new()ARROW-4172 - [Rust] more consistent naming in array buildersARROW-4174 - [Ruby] Add support for building composite array from raw Ruby objectsARROW-4175 - [GLib] Add support for decimal compare operatorsARROW-4177 - [C++] Add ThreadPool and TaskGroup microbenchmarksARROW-4183 - [Ruby] Add Arrow::Struct as an element of Arrow::StructArrayARROW-4184 - [Ruby] Add Arrow::RecordBatch#to_tableARROW-4191 - [C++] Use same CC and AR for jemalloc as for the main sourcesARROW-4199 - [GLib] Add garrow_seekable_input_stream_peek()ARROW-4207 - [Gandiva] [GLib] Add support for IfNodeARROW-4210 - [Python] Mention boost-cpp directly in the conda meta.yaml for pyarrowARROW-4211 - [GLib] Add GArrowFixedSizeBinaryDataTypeARROW-4214 - [Ruby] Add support for building RecordBatch from raw Ruby objectsARROW-4216 - [Python] Add CUDA API docsARROW-4228 - [GLib] Add garrow_list_data_type_get_field()ARROW-4229 - [Packaging] Set crossbow target explicitly to enable building arbitrary arrow repoARROW-4233 - [Packaging] Create a Dockerfile to build source archiveARROW-4239 - [Release] Updating .deb package names in the prepare script failed to run on OSXARROW-4240 - [Packaging] Documents for Plasma GLib and Gandiva GLib are missing in source archiveARROW-4241 - [Packaging] Disable crossbow conda OSX clang buildsARROW-4243 - [Python] Test failure with pandas 0.24.0rc1ARROW-4249 - [Plasma] Remove reference to logging.h from plasma/common.hARROW-4257 - [Release] Update release verification script to check binaries on BintrayARROW-4266 - [Python][CI] Disable ORC tests in dask integration testARROW-4269 - [Python] AttributeError: module 'pandas.core' has no attribute 'arrays'ARROW-4270 - [Packaging][Conda] Update xcode version and remove toolchain buildsARROW-4276 - [Release] Remove needless Bintray authentication from binaries verify scriptARROW-4306 - [Release] Update website and add blog post announcing 0.12.0 releasePARQUET-690 - [C++] Investigate / improve performance of Thrift utilitiesPARQUET-1271 - [C++] "parquet_reader" should be "parquet-reader"PARQUET-1439 - [C++] Parquet build fails when PARQUET_ARROW_LINKAGE is staticPARQUET-1449 - [C++] Can't build with ARROW_BOOST_VENDORED=ONPARQUET-1463 - [C++] Utilize revamped common hashing machinery for dictionary encodingPARQUET-1467 - [C++] Remove ChunkedAllocator code, now unusedPARQUET-1473 - [C++] Add helper function that converts ParquetVersion to human-friendly stringPARQUET-1484 - [C++] Improve memory usage of FileMetaDataBuilderBug Fixes
ARROW-1847 - [Doc] Document the difference between RecordBatch and Table in an FAQ fashionARROW-2026 - [Python] Cast all timestamp resolutions to INT96 use_deprecated_int96_timestamps=TrueARROW-2038 - [Python] Follow-up bug fixes for s3fs Parquet supportARROW-2113 - [Python] Incomplete CLASSPATH with "hadoop" contained in it can fool the classpath setting HDFS logicARROW-2591 - [Python] Segmentation fault when writing empty ListType column to ParquetARROW-2592 - [Python] Error reading old Parquet file due to metadata backwards compatibility issueARROW-2654 - [Python] Error with errno 22 when loading 3.6 GB Parquet fileARROW-2708 - [C++] Internal GetValues function in arrow::compute should check for nullptrARROW-2831 - [Plasma] MemoryError in teardownARROW-2970 - [Python] NumPyConverter::Visit for Binary/String/FixedSizeBinary can overflowARROW-2987 - [Python] test_cython_api can fail if run in an environment where vsvarsall.bat has been run more than onceARROW-3048 - [Python] Import pyarrow fails if scikit-learn is installed from conda (boost-cpp / libboost issue)ARROW-3058 - [Python] Feather reads fail with unintuitive error when conversion from pandas yields ChunkedArrayARROW-3186 - [GLib] mesonbuild failures in Travis CIARROW-3202 - [C++] Build does not succeed on Alpine LinuxARROW-3225 - [C++/Python] Pandas object conversion of ListType and ListType
ARROW-3324 - [Parquet] Free more internal resources when writing multiple row groupsARROW-3343 - [Java] Java tests fail non-deterministically with memory leak from Flight testsARROW-3405 - [Python] Document CSV readerARROW-3428 - [Python] from_pandas gives incorrect results when converting floating point to boolARROW-3436 - [C++] Boost version required by Gandiva is too new for Ubuntu 14.04ARROW-3437 - [Gandiva][C++] Configure static linking of libgcc, libstdc++ with LDFLAGSARROW-3438 - [Packaging] Escaped bulletpoints in changelogARROW-3445 - [GLib] Parquet GLib doesn't link Arrow GLibARROW-3449 - [C++] Support CMake 3.2 for "out of the box" buildsARROW-3466 - [Python] Crash when importing tensorflow and pyarrowARROW-3467 - Building against external double conversion is brokenARROW-3470 - [C++] Row-wise conversion tutorial has fallen out of dateARROW-3477 - [C++] Testsuite fails on 32 bit archARROW-3480 - [Website] Install document for Ubuntu is brokenARROW-3483 - [CI] Python 3.6 build failure on Travis-CIARROW-3485 - [C++] Examples fail with Protobuf errorARROW-3494 - [C++] re2 conda-forge package not working in toolchainARROW-3498 - [R] Make IPC APIs consistentARROW-3516 - [C++] Use unsigned type for difference of pointers in parallel_memcpyARROW-3517 - [C++] MinGW 32bit build causes g++ segvARROW-3524 - [C++] Fix compiler warnings from ARROW-3409 on clang-6ARROW-3527 - [R] Unused variables in R-package C++ codeARROW-3528 - [R] Typo in R documentationARROW-3535 - [Python] pip install tensorflow install too new numpy in manylinux1 buildARROW-3541 - [Rust] Update BufferBuilder to allow for new bit-packed BooleanArrayARROW-3544 - [Gandiva] Populate function registry in multiple compilation units to mitigate long compile times in release modeARROW-3549 - [Rust] Replace i64 with usize for some bit utility functionsARROW-3573 - [Rust] with_bitset does not set valid bits correctlyARROW-3580 - [Gandiva][C++] Build error with g++ 8.2.0ARROW-3586 - [Python] Segmentation fault when converting empty table to pandas with categoricalsARROW-3598 - [Plasma] plasma_store_server fails linking with GPU enabledARROW-3613 - [Go] Resize does not correctly update the lengthARROW-3613 - [Go] Resize does not correctly update the lengthARROW-3614 - [R] Handle Type::TIMESTAMP from Arrow to RARROW-3634 - [GLib] cuda.cpp compile errorARROW-3637 - [Go] Implement Stringer for arraysARROW-3658 - [Rust] validation of offsets buffer is incorrect for `List`
ARROW-3670 - [C++] Use FindBacktrace to find execinfo.h supportARROW-3687 - [Rust] Anything measuring array slots should be `usize`ARROW-3698 - [C++] Segmentation fault when using a large table in GandivaARROW-3700 - [C++] CSV parser should allow ignoring empty linesARROW-3703 - [Python] DataFrame.to_parquet crashes if datetime column has time zonesARROW-3704 - [Gandiva] Can't build with g++ 8.2.0ARROW-3707 - [C++] test failure with zstd 1.3.7ARROW-3711 - [C++] Don't pass CXX_FLAGS to C_FLAGSARROW-3712 - [CI] License check regression (RAT failure)ARROW-3715 - [C++] gflags_ep fails to build with CMake 3.13ARROW-3716 - [R] Missing cases for ChunkedArray conversionARROW-3728 - [Python] Merging Parquet Files - Pandas Meta in Schema MismatchARROW-3734 - [C++] Linking static zstd library fails on Arch x86-64ARROW-3740 - [C++] Calling ArrayBuilder::Resize with length smaller than current appended length results in invalid stateARROW-3742 - Fix pyarrow.types & gandiva cython bindingsARROW-3745 - [C++] CMake passes static libraries multiple times to linkerARROW-3754 - [Packaging] Zstd configure error on linux package buildsARROW-3756 - [CI/Docker/Java] Java tests are failing in docker-compose setupARROW-3765 - [Gandiva] Segfault when the validity bitmap has not been allocatedARROW-3766 - [Python] pa.Table.from_pandas doesn't use schema orderingARROW-3768 - [Python] set classpath to hdfs not hadoop executableARROW-3775 - [C++] Handling Parquet Arrow reads that overflow a BinaryArray capacityARROW-3790 - [C++] Signed to unsigned integer cast yields incorrect results when type sizes are the sameARROW-3792 - [Python] Segmentation fault when writing empty RecordBatches to ParquetARROW-3793 - [C++] TestScalarAppendUnsafe is not testing unsafe appendsARROW-3797 - [Rust] BinaryArray::value_offset incorrect in offset caseARROW-3805 - [Gandiva] handle null validity bitmap in if-else expressionsARROW-3831 - [C++] arrow::util::Codec::Decompress() doesn't return decompressed data sizeARROW-3835 - [C++] arrow::io::CompressedOutputStream::raw() impementation is missingARROW-3837 - [C++] gflags link errors on WindowsARROW-3866 - [Python] Column metadata is not transferred to tables in pyarrowARROW-3869 - [Rust] "invalid fastbin errors" since Rust nightly-2018-11-03ARROW-3874 - [Gandiva] Cannot build: LLVM not detected correctlyARROW-3879 - [C++] cuda-test failureARROW-3888 - [C++] Compilation warnings with gcc 7.3.0ARROW-3889 - [Python] creating schema with invalid paramaters causes segmanetation faultARROW-3890 - [Python] Creating Array with explicit string type fails on Python 2.7ARROW-3894 - [Python] Error reading IPC file with no record batchesARROW-3898 - parquet-arrow example has compilation errorsARROW-3909 - [Python] Table.from_pandas call that seemingly should zero copy does notARROW-3918 - [Python] ParquetWriter.write_table doesn't support coerce_timestamps or allow_truncated_timestampsARROW-3920 - Plasma reference counting not properly done in TensorFlow custom operator.ARROW-3931 - [C++] Make possible to build regardless of LANGARROW-3936 - Add _O_NOINHERIT to the file open flags on WindowsARROW-3937 - [Rust] Rust nightly build is failingARROW-3940 - [Python/Documentation] Add required packages to the development instructionARROW-3941 - [R] RecordBatchStreamReader$schemaARROW-3942 - [R] Feather api fixesARROW-3953 - Compat with pandas 0.24 rename of MultiIndex labels -> codesARROW-3955 - [GLib] Add (transfer full) to free when no longer neededARROW-3957 - [Python] Better error message when user connects to HDFS cluster with wrong portARROW-3961 - [Python/Documentation] Fix wrong path in the pyarrow READMEARROW-3969 - [Rust] CI build broken because rustfmt not available on nightly toolchainARROW-3976 - [Ruby] Homebrew donation solicitation on CLI breaking CI buildsARROW-3977 - [Gandiva] gandiva cpp tests not running in CIARROW-3979 - [Gandiva] fix all valgrind reported errorsARROW-3980 - [C++] Fix CRTP use in json-simple.ccARROW-3989 - [Rust] CSV reader should handle case sensitivity for boolean valuesARROW-3996 - [C++] Insufficient description on buildARROW-4008 - [C++] Integration test executable failureARROW-4011 - [Gandiva] Refer irhelpers.bc in build directoryARROW-4019 - [C++] Fix coverity issuesARROW-4033 - [C++] thirdparty/download_dependencies.sh uses tools or options not available in older LinuxesARROW-4034 - [Ruby] Interface for FileOutputStream doesn't respect append=TrueARROW-4041 - [CI] Python 2.7 run uses Python 3.6ARROW-4049 - [C++] Arrow never use glog even though glog is linked.ARROW-4052 - [C++] Linker errors with glog and gflagsARROW-4053 - [Python/Integration] HDFS Tests failing with I/O operation on closed fileARROW-4055 - [Python] Fails to convert pytz.utc with versions 2018.3 and earlierARROW-4058 - [C++] arrow-io-hdfs-test fails when run against HDFS cluster from docker-composeARROW-4065 - [C++] arrowTargets.cmake is brokenARROW-4066 - Instructions to create Sphinx documentationARROW-4070 - [C++] ARROW_BOOST_VENDORED doesn't work properly with ninja buildARROW-4073 - [Python] Parquet test failures on AppVeyorARROW-4074 - [Python] test_get_library_dirs_win32 fails if libraries installed someplace different from conda or wheel packagesARROW-4078 - [CI] Run Travis job where documentation is built when docs/ is changedARROW-4088 - [Python] Table.from_batches() fails when passed a schema with metadataARROW-4089 - [Plasma] The tutorial is wrong regarding the parameter type of PlasmaClient.CreateARROW-4101 - [C++] Binary identity cast not implementedARROW-4106 - [Python] Tests fail to run because hypothesis update broke its APIARROW-4109 - [Packaging] Missing glog dependency from arrow-cpp conda recipeARROW-4113 - [R] Version number patch broke buildARROW-4114 - [C++][DOCUMENTATION] Add "python" to Linux build instructionsARROW-4115 - [Gandiva] valgrind complains that boolean output data buffer has uninited dataARROW-4118 - [Python] Error with "asv run"ARROW-4125 - [Python] ASV benchmarks fail to run if Plasma extension is not built (e.g. on Windows)ARROW-4126 - [Go] offset not used when accessing boolean arrayARROW-4128 - [C++][DOCUMENTATION] Update style guide to reflect some more exceptionsARROW-4130 - [Go] offset not used when accessing binary arrayARROW-4134 - [Packaging] Properly setup timezone in docker tests to prevent ORC adapter's abortARROW-4135 - [Python] Can't reload a pandas dataframe containing a list of datetime.timeARROW-4137 - [Rust] Move parquet code into a separate crateARROW-4138 - [Python] setuptools_scm customization does not work for versions above 0.9.0 on WindowsARROW-4147 - [JAVA] Reduce heap usage for variable width vectorsARROW-4149 - [CI/C++] Parquet test misses ZSTD compression codec in CMake 3.2 nightly buildsARROW-4157 - [C++] -Wdocumentation failures with clang 6.0 on Ubuntu 18.04ARROW-4171 - [Rust] fix parquet crate release versionARROW-4173 - JIRA library name is wrong in error message of dev/merge_arrow_pr.pyARROW-4178 - [C++] Fix TSan and UBSan errorsARROW-4179 - [Python] Tests crashing on all platforms in CIARROW-4182 - [Python][CI] SEGV frequencyARROW-4185 - [Rust] Appveyor builds are brokenARROW-4186 - [C++] BitmapWriters clobber the first byte when length=0ARROW-4188 - [Rust] There should be a README in the top level rust directoryARROW-4197 - [C++] Emscripten compiler fails building ArrowARROW-4200 - [C++] conda_env_* files cannot be used to create a fresh conda environment on WindowsARROW-4209 - [Gandiva] returning IR structs causes issues with windowsARROW-4215 - [GLib] Fix typos in documentationARROW-4227 - [GLib] Field in composite data type returns wrong data typeARROW-4237 - [Packaging] Fix CMAKE_INSTALL_LIBDIR in release verification scriptARROW-4238 - [Packaging] Fix RC version conflict between crossbow and rakeARROW-4246 - [Plasma][Python] PlasmaClient.list doesn't work with CUDA enabled PlasmaARROW-4246 - [Plasma][Python] PlasmaClient.list doesn't work with CUDA enabled PlasmaARROW-4256 - [Release] Update Windows verification script for 0.12 releaseARROW-4258 - [Python] Safe cast fails from numpy float64 array with nans to integerARROW-4260 - [Python] test_serialize_deserialize_pandas is failing in multiple build entriesPARQUET-1426 - [C++] parquet-dump-schema has poor usabilityPARQUET-1458 - [C++] parquet::CompressionToString not recognizing brotli compressionPARQUET-1469 - [C++] DefinitionLevelsToBitmap can overwrite prior decoded dataPARQUET-1471 - [C++] Out of bounds access in statistics UpdateSpaced when writing optional list with null list slotsPARQUET-1481 - [C++] SEGV when reading corrupt parquet fileApache Arrow 0.11.1 (2018-10-23)
New Features and Improvements
ARROW-3353 - [Packaging] Build python 3.7 wheelsARROW-3534 - [Python] Update zlib library in manylinux1 imageARROW-3546 - [Python] Provide testing setup to verify wheel binaries work in one or more common Linux distributionsARROW-3565 - [Python] Pin tensorflow to 1.11.0 in manylinux1 container
Bug Fixes
ARROW-3514 - [Python] zlib deflate exception when writing Parquet fileARROW-3907 - [Python] from_pandas errors when schemas are used with lower resolution timestamps
Apache Arrow 0.11.0 (2018-10-08)
New Features and Improvements
ARROW-25 - [C++] Implement delimited file scanner / CSV readerARROW-249 - [Flight] Define GRPC IDL / wire protocol for messaging with Arrow dataARROW-614 - [C++] Use glog (or some other tool) to print stack traces in debug builds on errorsARROW-1325 - [R] Bootstrap R bindings subprojectARROW-1424 - [Python] Initial bindings for libarrow_gpuARROW-1491 - [C++] Add casting implementations from strings to numbers or booleanARROW-1521 - [C++] Add Reset method to BufferOutputStream to enable object reuseARROW-1563 - [C++] Implement logical unary and binary kernels for boolean arraysARROW-1563 - [C++] Implement logical unary and binary kernels for boolean arraysARROW-1860 - [C++] Add data structure to "stage" a sequence of IPC messages from in-memory dataARROW-1949 - [Python/C++] Add option to Array.from_pandas and pyarrow.array to perform unsafe castsARROW-1963 - [C++/Python] Create Array from sequence of numpy.datetime64ARROW-1968 - [Python] Unit testing setup for ORC filesARROW-2165 - enhance AllocatorListener to listen for child allocator addition and removalARROW-2338 - [Scripts] Windows release verification script should create a conda environmentARROW-2352 - [C++/Python] Test OSX packaging in Travis matrixARROW-2519 - [Rust] Implement min/max for primitive arraysARROW-2520 - [Rust] CI should also build against nightly RustARROW-2555 - [Python] Provide an option to convert on coerce_timestamps instead of errorARROW-2583 - [Rust] Buffer should be typelessARROW-2617 - [Rust] Schema should contain fields not columnsARROW-2687 - [JS] Example usage in README is outdatedARROW-2734 - [Python] Cython api example doesn't work by default on macOSARROW-2750 - [MATLAB] Add MATLAB support for reading numeric types from Feather filesARROW-2799 - [Python] Add safe option to Table.from_pandas to avoid unsafe castsARROW-2813 - [C++] Strip uninformative lcov output from Travis CI logsARROW-2813 - [C++] Strip uninformative lcov output from Travis CI logsARROW-2817 - [C++] Enable libraries to be installed in msys2 on WindowsARROW-2840 - [C++] See if stream alignment logic can be simplifiedARROW-2865 - [C++/Python] Reduce some duplicated code in python/builtin_convert.ccARROW-2889 - [C++] Add optional argument to ADD_ARROW_TEST CMake function to add unit test prefixARROW-2900 - [Python] Improve performance of appending nested NumPy arrays in builtin_convert.ccARROW-2936 - [Python] Implement Table.cast for casting from one schema to another (if possible)ARROW-2948 - [Packaging] Generate changelog with crossbowARROW-2950 - [C++] Clean up util/bit-util.hARROW-2952 - [C++] Dockerfile for running include-what-you-use checksARROW-2958 - [C++] Flatbuffers EP fails to compile with GCC 8.1ARROW-2960 - [Packaging] Fix verify-release-candidate for binary packages and fix release cutting script for lib64 cmake issueARROW-2964 - [Go] wire all currently implemented array types in array.MakeFromDataARROW-2971 - [Python] Give more descriptive names to python_to_arrow.cc/arrow_to_python.ccARROW-2972 - [Python] Implement inference logic for uint64 conversions in builtin_convert.ccARROW-2975 - [Plasma] TensorFlow op: Compilation only working if arrow found by pkg-configARROW-2976 - [Python] Directory in pyarrow.get_library_dirs() on Travis doesn't contain libarrow.soARROW-2979 - [GLib] Add operator functions in GArrowDecimal128ARROW-2983 - [Packaging] Verify source release and binary artifacts in different scriptsARROW-2989 - [C++] Remove deprecated APIs in 0.10.0 and belowARROW-2991 - [CI] Cut down number of AppVeyor jobsARROW-2994 - [C++] Only include Python C header directories for Python-related compilation unitsARROW-2996 - [C++] Fix typo in cpp/.clang-tidyARROW-2998 - [C++] Add variants of AllocateBuffer, AllocateResizeableBuffer that return unique_ptr
ARROW-2999 - [Python] Do not run ASV benchmarks in every Travis CI build to improve runtimesARROW-3000 - [Python] Do not build unit tests other than python-test in travis_script_python.shARROW-3001 - [Packaging] Don't modify PATH during rust release verificationARROW-3002 - [Python] Implement better DataType hash functionARROW-3003 - [Doc] Enable Java doc in dev/gen_apidocs/create_documents.shARROW-3005 - [Website] Update website and write blog post for 0.10.0 release announcementARROW-3008 - [Packaging] Verify GPU related modules if availableARROW-3009 - [Python] pyarrow.orc uses APIs now prohibited in 0.10.0ARROW-3010 - [GLib] Update README to use BundlerARROW-3017 - [C++] Don't throw exception in arrow/util/thread-pool.hARROW-3018 - [Plasma] Improve random ObjectID generationARROW-3018 - [Plasma] Improve random ObjectID generationARROW-3019 - [Packaging] Use Bundler to verify Arrow GLibARROW-3021 - [Go] support for ListARROW-3022 - [Go] support for StructARROW-3023 - [C++] Use gold linker in builds if it is availableARROW-3024 - [C++] Replace usages of std::mutex with atomics in memory_pool.ccARROW-3025 - [C++] Add option to switch between dynamic and static linking in unit test executablesARROW-3026 - [Plasma] Only run Plasma Python unit tests under valgrind once instead of twice in CIARROW-3027 - [Ruby] Stop "git tag" by "rake release"ARROW-3028 - [Python] Trim unneeded work from documentation build in Travis CIARROW-3029 - [Python] pkg_resources is slowARROW-3031 - [Go] Streamline release of Arrays and BuildersARROW-3033 - [Dev] docker-compose test tooling does not seem to cache built Docker imagesARROW-3034 - [Packaging] Source archive can't be extracted by bsdtar on MSYS2ARROW-3035 - [Rust] Examples in README.md do not runARROW-3036 - [Go] add support for slicing ArraysARROW-3037 - [Go] add support NullArrayARROW-3042 - [Go] add badge to GoDoc in the Go-Arrow READMEARROW-3043 - [C++] pthread doesn't exist on MinGWARROW-3044 - [Python] Remove all occurrences of cython's legacy property definition syntaxARROW-3045 - [Python] Remove nullcheck from ipc Message and MessageReaderARROW-3046 - [GLib] Use rubyish method in test-orc-file-reader.rbARROW-3050 - [C++] Adopt HiveServer2 client C++ codebaseARROW-3051 - [C++] Status performance optimization from Impala/KuduARROW-3057 - [INTEGRATION] Fix spark and hdfs dockerfilesARROW-3059 - [C++] Streamline namespace array::testARROW-3060 - [C++] Factor out parsing routinesARROW-3062 - [Python] Extend fast libtensorflow_framework.so compatibility workaround to Python 2.7ARROW-3064 - [C++] Add option to ADD_ARROW_TEST to indicate additional dependencies for particular unit test executablesARROW-3067 - [Packaging] Support dev/rc/release .deb/.rpm buildsARROW-3068 - [Packaging] Bump version to 0.11.0-SNAPSHOTARROW-3069 - [Release] Stop using SHA1 checksums per ASF policyARROW-3072 - [C++] Use ARROW_RETURN_NOT_OK instead of RETURN_NOT_OK in header filesARROW-3075 - [C++] Incorporate apache/parquet-cpp codebase into Arrow C++ codebase and build systemARROW-3076 - [Website] Add Google Analytics tags to C++, Python API docsARROW-3088 - [Rust] Use internal `Result` type instead of `Result`
ARROW-3090 - [Rust] Accompany error messages with assertionsARROW-3094 - [Python] Allow lighter construction of pa.Schema / pa.StructTypeARROW-3099 - [C++] Add benchmark for number parsingARROW-3105 - [Plasma] Improve flushing error messageARROW-3106 - [Website] Update committers and PMC roster on websiteARROW-3109 - [Python] Add Python 3.7 virtualenvs to manylinux1 containerARROW-3110 - [C++] Compilation warnings with gcc 7.3.0ARROW-3111 - [Java] Enable changing default logging level when running testsARROW-3114 - [Website] Add information about user@ mailing list to website / Community pageARROW-3115 - [Java] Style Checks - Fix import orderingARROW-3116 - [Plasma] Add "ls" to object storeARROW-3117 - [GLib] Add garrow_chunked_array_to_string()ARROW-3119 - [Packaging] Nightly packaging script failsARROW-3127 - [C++] Add Tutorial about Sending Tensor from C++ to PythonARROW-3128 - [C++] Support system shared zlibARROW-3129 - [Packaging] Stop to use deprecated BuildRoot and Group in .rpmARROW-3130 - [Go] add initial support for Go modulesARROW-3136 - [C++] Clean up arrow:: public APIARROW-3142 - [C++] Fetch all libs from toolchain environmentARROW-3143 - [C++] CopyBitmap into existing memoryARROW-3146 - [C++] Barebones Flight RPC server and client implementationsARROW-3147 - [C++] MSVC version isn't detected in code page 932ARROW-3148 - [C++] MSVC shows C4819 warning on code page 932ARROW-3152 - [C++][Packaging] Use dynamic linking for zlib in conda recipesARROW-3153 - [Packaging] Fix broken nightly package builds introduced with recent cmake changes and orc testsARROW-3157 - [C++] Improve buffer creation for typed dataARROW-3158 - [C++] Handle float truncation during castingARROW-3160 - [Python] Improve pathlib.Path support in parquet and filesystem modulesARROW-3163 - [Python] Cython dependency is missing in non wheel packageARROW-3167 - [CI] Limit clcache cache sizeARROW-3168 - [C++] Restore pkgconfig for Parquet C++ librariesARROW-3170 - [C++] Implement "readahead spooler" class for background input bufferingARROW-3171 - [Java] checkstyle - fix line length and indentationARROW-3172 - [Rust] Update documentation for datatypes.rsARROW-3174 - [Rust] run examples as part of CIARROW-3177 - [Rust] Update expected error messages for tests that 'should panic'ARROW-3180 - [C++] Add docker-compose setup to simulate Travis CI run locallyARROW-3181 - [Packaging] Adjust conda package scripts to account for Parquet codebase migrationARROW-3182 - [C++] Merge Gandiva codebaseARROW-3187 - [Plasma] Make Plasma Log pluggable with glogARROW-3195 - [C++] NumPy initialization error check is missing in testARROW-3196 - Enable merge_arrow_py.py script to merge Parquet patches and set fix versionsARROW-3197 - [C++] Add instructions to cpp/README.md about Parquet-only development and Arrow+ParquetARROW-3198 - [Website] Blog post for 0.11 releaseARROW-3211 - [C++] gold linker doesn't work with MinGW-w64ARROW-3212 - [C++] Create deterministic IPC metadataARROW-3213 - [C++] Use CMake to build vendored Snappy on WindowsARROW-3214 - [C++] Disable insecure warnings with MinGW buildARROW-3215 - [C++] Add support for finding libpython on MSYS2ARROW-3216 - [C++] libpython isn't linked to libarrow_python in MinGW buildARROW-3217 - [C++] ARROW_STATIC definition is missing in MinGW buildARROW-3218 - [C++] Utilities has needless pthread link in MinGW buildARROW-3219 - [C++] Use Win32 API in MinGWARROW-3223 - [GLib] Use the same shared object versioning rule in C++ARROW-3229 - [Packaging]: Adjust wheel package scripts to account for Parquet codebase migrationARROW-3234 - [C++] Link order is wrong when ARROW_ORC=on and ARROW_PROTOBUF_USE_SHARED=ONARROW-3235 - [Packaging] Update deb namesARROW-3236 - [C++] OutputStream bookkeeping logic when writing IPC file format is incorrectARROW-3240 - [GLib] Add build instructions using MesonARROW-3242 - [C++] Use coarser-grained dispatch to SIMD hash functionsARROW-3249 - [Python] Run flake8 on integration_test.py and crossbow.pyARROW-3250 - [C++] Create Buffer implementation that takes ownership for the memory from a std::string via std::moveARROW-3252 - [C++] Do not hard code the "v" part of versions in thirdparty toolchainARROW-3257 - [C++] Stop to use IMPORTED_LINK_INTERFACE_LIBRARIESARROW-3258 - [GLib] CI is failued on macOSARROW-3259 - [GLib] Rename "writeable" to "writable"ARROW-3261 - [Python] Add "field" method to select fields from StructArrayARROW-3262 - [Python] Implement __getitem__ with integers on pyarrow.ColumnARROW-3264 - [Java] checkstyle - fix whitespaceARROW-3267 - [Python] Create empty table from schemaARROW-3268 - [CI] Reduce conda times on AppVeyorARROW-3269 - [Python] Fix warnings in unit test suiteARROW-3270 - [Release] Adjust release verification scripts to recent parquet migrationARROW-3274 - [Packaging] Missing glog dependency from conda-forge recipesARROW-3276 - [Packaging] Add support Parquet related Linux packagesARROW-3281 - [Java] Make sure that WritableByteChannel in WriteChannel writes out complete bytesARROW-3282 - [R] initial R functionalityARROW-3284 - [R] Adding R Error in StatusARROW-3285 - [GLib] Add arrow_cpp_build_type and arrow_cpp_build_dir Meson optionsARROW-3286 - [C++] ARROW_EXPORT for RecordBatchBuilder is missingARROW-3287 - [C++] "redeclared without dllimport attribute after being referenced with dll linkage" with MinGWARROW-3288 - [GLib] Add new API index for 0.11.0ARROW-3300 - [Release] Update .deb package names in preparationARROW-3301 - [Website] Update Jekyll and Bootstrap 4ARROW-3305 - [JS] Incorrect development documentation link in javascript readmeARROW-3309 - [JS] Missing links from DEVELOP.mdARROW-3313 - [R] Run clang-format, cpplint checks on R C++ codeARROW-3313 - [R] Run clang-format, cpplint checks on R C++ codeARROW-3319 - [GLib] Expose AlignStream methods in InputStream, OutputStream classesARROW-3320 - [C++] Improve float parsing performanceARROW-3321 - [C++] Improve integer parsing performanceARROW-3334 - [Python] Update conda packages to new numpy requirementARROW-3335 - [Python] Add ccache to manylinux1 containerARROW-3339 - [R] Support for character vectorsARROW-3341 - [R] Support for logical vectorARROW-3349 - [C++] Use aligned API in MinGWARROW-3350 - [Website] Fix powered by linksARROW-3352 - [Packaging] Fix recently failing wheel buildsARROW-3356 - [Python] Document parameters of Table.to_pandas methodARROW-3357 - [Rust] Add a mutable buffer implementationARROW-3360 - [GLib] Import Parquet bindingsARROW-3363 - [C++/Python] Add helper functions to detect scalar Python typesARROW-3371 - [Python] Remove check_metadata argument for Field.equals docstringARROW-3375 - [Rust] Remove memory_pool.rsARROW-3376 - [C++] Add double-conversion to cpp/thirdparty/download_dependencies.shARROW-3377 - [Gandiva][C++] Remove If statement from bit map set functionARROW-3382 - [C++] Run Gandiva tests in Travis CIARROW-3392 - [Python] Support filters in disjunctive normal form in ParquetDatasetARROW-3395 - [C++/Python] Add docker container for lintingARROW-3397 - [C++] Use relative CMake path for modulesARROW-3400 - [Packaging] Add support Parquet GLib related Linux packagesARROW-3404 - [C++] Make CSV chunker fasterARROW-3411 - [Packaging] dev/release/01-perform.sh doesn't have executable bitARROW-3412 - [Packaging] rat failure in dev/release/02-source.shARROW-3413 - [Packaging] dev/release/02-source.sh doesn't generate Parquet GLib documentARROW-3415 - [Packaging] dev/release/verify-release-cndidate.sh fails in "conda activate arrow-test"ARROW-3416 - [Packaging] dev/release/02-source.sh must use SHA512 instead of SHA1ARROW-3417 - [Packaging] dev/release/verify-release-cndidate.sh fails Parquet C++ testARROW-3418 - [C++] Update Parquet snapshot version for releaseARROW-3423 - [Packaging] Remove RC information from deb/rpmARROW-3443 - [Java] Flight reports memory leaks in TestBasicOperationPARQUET-169 - Parquet-cpp: Implement support for bulk reading and writing repetition/definition levels.PARQUET-267 - Detach thirdparty code from build configuration.PARQUET-416 - C++11, cpplint cleanup, package target and header installationPARQUET-418 - Add a utility to print contents of a Parquet file to stdoutPARQUET-428 - Support INT96 and FIXED_LEN_BYTE_ARRAY typesPARQUET-434 - Add a ParquetFileReader class to encapsulate some low-level details of interacting with Parquet filesPARQUET-435 - Provide vectorized ColumnReader interfacePARQUET-436 - Implement ParquetFileWriter class entry point for generating new Parquet filesPARQUET-437 - Incorporate googletest thirdparty dependency and add cmake tools (ADD_PARQUET_TEST) to simplify adding new unit testsPARQUET-438 - Update RLE encoder/decoder modules from Impala upstream changes and adapt unit testsPARQUET-439 - Conform all copyright headers to ASF requirementsPARQUET-442 - Convert flat SchemaElement vector to implied nested schema data structurePARQUET-448 - Add cmake option to skip building the unit testsPARQUET-449 - Update to latest parquet.thriftPARQUET-451 - Add a RowGroup reader interface classPARQUET-456 - Add zlib codec supportPARQUET-463 - Add DCHECK* macros for assertions in debug buildsPARQUET-468 - Add a cmake option to generate the Parquet thrift headers with the thriftc in the environmentPARQUET-477 - Enable clang-format check during the Travis CI buildPARQUET-482 - Organize src code file structure to have a very clear folder with public headers.PARQUET-485 - Decouple data page delimiting from column reader / scanner classes, create test fixturesPARQUET-488 - Add SSE-related cmake options to manage compiler flagsPARQUET-489 - Add visibility macros to be used for public and internal APIs of libparquetPARQUET-494 - Implement PLAIN_DICTIONARY encoding and decodingPARQUET-496 - Fix cpplint configuration to be more restrictivePARQUET-497 - Decouple Parquet physical file structure from FileReader classPARQUET-499 - Complete PlainEncoder implementation for all primitive types and test end to endPARQUET-501 - Add an OutputStream abstraction (capable of memory allocation) for Encoder public APIPARQUET-503 - Re-enable parquet 2.0 encodingsPARQUET-508 - Add ParquetFilePrinterPARQUET-508 - Add ParquetFilePrinterPARQUET-512 - Add optional google/benchmark 3rd-party dependency for performance testingPARQUET-515 - Add "Reset" to LevelEncoder and LevelDecoderPARQUET-518 - Review usages of size_t and unsigned integers generally per Google style guidePARQUET-519 - Disable compiler warning supressions and fix all DEBUG build warningsPARQUET-520 - Add version of LocalFileSource that uses memory-mapping for zero-copy readsPARQUET-533 - Simplify RandomAccessSource API to combine Seek/ReadPARQUET-538 - Improve ColumnReader TestsPARQUET-542 - Support memory allocation from external memoryPARQUET-545 - Improve API to support Decimal typePARQUET-547 - Refactor most templates to use DataType structs rather than the Type::type enumPARQUET-551 - Handle compiler warnings due to disabled DCHECKs in release buildsPARQUET-556 - Extend RowGroupStatistics to include "min" "max" statisticsPARQUET-559 - Enable InputStream as a source to the ParquetFileReaderPARQUET-564 - Add option to run unit tests with valgrind --tool=memcheckPARQUET-566 - Add method to retrieve the full column pathPARQUET-568 - Read only specified top-level columns in DebugPrintPARQUET-572 - Rename parquet_cpp namespace to parquetPARQUET-573 - C++: Create a public API for reading and writing file metadataPARQUET-582 - Conversion functions for Parquet enums to Thrift enumsPARQUET-583 - Implement Parquet to Thrift schema conversionPARQUET-587 - Implement BufferReader::Read(int64_t,uint8_t*)PARQUET-589 - Implement Chunked InMemoryInputStream for better memory usagePARQUET-592 - Support compressed writesPARQUET-593 - Add API for writing Page statisticsPARQUET-595 - Add API for key-value metadataPARQUET-595 - Add API for key-value metadataPARQUET-597 - Add data rates to benchmark outputPARQUET-598 - [C++] Test writing all primitive data typesPARQUET-600 - Add benchmarks for RLE-Level encodingPARQUET-603 - Implement missing information in schema descriptorPARQUET-605 - Expose schema node in ColumnDescriptorPARQUET-607 - Public Writer headerPARQUET-610 - Print ColumnMetaData for each RowGroupPARQUET-616 - C++: WriteBatch should accept const arraysPARQUET-619 - C++: Add OutputStream for local filesPARQUET-625 - Improve RLE read performancePARQUET-633 - Add version to WriterPropertiesPARQUET-634 - Consistent private linking of dependenciesPARQUET-636 - Expose selection for different encodingsPARQUET-641 - Instantiate stringstream only if needed in SerializedPageReader::NextPagePARQUET-646 - [C++] Enable easier 3rd-party toolchain clang builds on LinuxPARQUET-666 - PLAIN_DICTIONARY write supportPARQUET-671 - Improve performance of RLE/bit-packed decoding in parquet-cppPARQUET-679 - [C++] Build and unit tests support for MSVC on WindowsPARQUET-679 - [C++] Build and unit tests support for MSVC on WindowsPARQUET-679 - [C++] Build and unit tests support for MSVC on WindowsPARQUET-679 - [C++] Build and unit tests support for MSVC on WindowsPARQUET-681 - Add tool to scan a parquet filePARQUET-681 - Add tool to scan a parquet filePARQUET-687 - C++: Switch to PLAIN encoding if dictionary grows too largePARQUET-689 - C++: Compress DataPages eagerlyPARQUET-699 - Update parquet.thrift from https://github.com/apache/parquet-format PARQUET-712 - C++: Read into Arrow memoryPARQUET-721 - Performance benchmarks for reading into Arrow structuresPARQUET-724 - Test more advanced properties settingPARQUET-728 - [C++] Bring parquet::arrow up to date with API changes in arrow::ioPARQUET-728 - [C++] Bring parquet::arrow up to date with API changes in arrow::ioPARQUET-731 - [CPP] Add API to return metadata size and Skip reading valuesPARQUET-737 - Use absolute namespace in macrosPARQUET-752 - [C++] Conform parquet_arrow to upstream API changesPARQUET-762 - C++: Use optimistic allocation instead of Arrow BuildersPARQUET-763 - C++: Expose ParquetFileReader through Arrow readerPARQUET-769 - C++: Add support for Brotli CompressionPARQUET-778 - Standardize the schema output to match the parquet-mr formatPARQUET-782 - C++: Support writing to Arrow sinksPARQUET-785 - C++: List conversion for Arrow SchemasPARQUET-805 - C++: Read Int96 into Arrow Timestamp(ns)PARQUET-807 - [C++] Add API to read file metadata only from a file handlePARQUET-807 - [C++] Add API to read file metadata only from a file handlePARQUET-809 - [C++] Add API to determine if two files' schemas are compatiblePARQUET-813 - C++: Build dependencies using CMake External projectPARQUET-820 - C++: Decoders should directly emit arrays with spacing for null entriesPARQUET-829 - C++: Make use of ARROW-469PARQUET-830 - [C++] Add additional configuration options to parquet::arrow::OpenFIlePARQUET-833 - C++: Provide API to write spaced arrays (e.g. Arrow)PARQUET-834 - C++: Support r/w of arrow::ListArrayPARQUET-835 - [C++] Add option to parquet::arrow to read columns in parallel using a thread poolPARQUET-836 - [C++] Add column selection to parquet::arrow::FileReaderPARQUET-844 - [C++] Consolidate encodings, schema, and compression subdirectories into fewer filesPARQUET-848 - [C++] Consolidate libparquet_thrift subcomponentPARQUET-857 - [C++] Flatten parquet/encodings directoryPARQUET-858 - [C++] Flatten parquet/column directory, consolidate related codePARQUET-859 - [C++] Flatten parquet/file directoryPARQUET-862 - Provide defaut cache size values if CPU info probing is not availablePARQUET-866 - [C++] Account for API changes in ARROW-33PARQUET-867 - [C++] Support writing sliced Arrow arraysPARQUET-874 - [C++] Use default memory allocator from ArrowPARQUET-877 - C++: Update Arrow Hash, update Version in metadata.PARQUET-882 - [CPP] Improve Application Version parsingPARQUET-890 - C++: Support I/O of DATE columns in parquet_arrowPARQUET-894 - Fix compilation warningPARQUET-894 - Fix compilation warningPARQUET-897 - [C++] Only use designated public headers from libarrowPARQUET-903 - C++: Add option to set RPATH to ORIGINPARQUET-909 - [CPP]: Reduce buffer allocations (mallocs) on critical pathPARQUET-909 - [CPP]: Reduce buffer allocations (mallocs) on critical pathPARQUET-911 - C++: Support nested structs in parquet_arrowPARQUET-928 - [C++] Support pkg-configPARQUET-929 - [C++] Handle arrow::DictionaryArray when writing Arrow dataPARQUET-930 - [C++] Account for all Arrow date/time typesPARQUET-934 - [C++] Support multiarch on DebianPARQUET-935 - [C++] Set shared library version for .deb packagesPARQUET-946 - [C++] Refactoring in parquet::arrow::FileReader to be able to read a single row groupPARQUET-953 - [C++] Change arrow::FileWriter API to be initialized from a Schema, and provide for writing multiple tablesPARQUET-967 - [C++] Combine libparquet/libparquet_arrow librariesPARQUET-970 - Add Add Lz4 and Zstd compression codecsPARQUET-978 - [C++] Minimizing footer reads for small(ish) metadataPARQUET-984 - C++: Add abi and so version to pkg-configPARQUET-991 - [C++] Fix compiler warnings on MSVC and build with /WX in AppveyorPARQUET-991 - [C++] Fix compiler warnings on MSVC and build with /WX in AppveyorPARQUET-991 - [C++] Fix compiler warnings on MSVC and build with /WX in AppveyorPARQUET-999 - Improve MSVC build - Enable PARQUET_BUILD_BENCHMARKSPARQUET-1008 - Update TypedColumnReader::ReadBatch method to accept batch_size as int64_tPARQUET-1035 - Write Int96 from Arrow Timestamp(ns)PARQUET-1037 - Allow final RowGroup to be unfilledPARQUET-1041 - C++: Support Arrow's NullArrayPARQUET-1043 - [C++] Raise minimum supported CMake version to 3.2PARQUET-1044 - [C++] Use compression libraries from Apache ArrowPARQUET-1045 - [C++] Refactor to account for computational utility code migration in ARROW-1154PARQUET-1053 - Fix unused result warnings due to unchecked StatusesPARQUET-1053 - Fix unused result warnings due to unchecked StatusesPARQUET-1068 - [C++] Use more vanilla Google C++ code formattingPARQUET-1068 - [C++] Use more vanilla Google C++ code formattingPARQUET-1072 - [C++] Add ARROW_NO_DEPRECATED_API to CI to check for deprecated API usePARQUET-1078 - [C++] Add Arrow writer option to coerce timestamps to milliseconds or microsecondsPARQUET-1079 - [C++] Account for Arrow API change in ARROW-1335PARQUET-1083 - [C++] Refactor core logic in parquet-scan.cc so that it can be used as a library function for benchmarkingPARQUET-1083 - [C++] Refactor core logic in parquet-scan.cc so that it can be used as a library function for benchmarkingPARQUET-1086 - [C++] Remove usage of arrow/util/compiler-util.h after 1.3.0 releasePARQUET-1087 - [C++] Add wrapper for ScanFileContents in parquet::arrow that catches exceptionsPARQUET-1092 - [C++] Write Arrow tables with chunked columnsPARQUET-1093 - C++: Improve Arrow level generation error messagePARQUET-1094 - C++: Add benchmark for boolean Arrow column I/OPARQUET-1095 - [C++] Read and write Arrow decimal valuesPARQUET-1104 - [C++] Upgrade to Apache Arrow 0.7.0 RC0PARQUET-1150 - C++: Hide statically linked boost symbolsPARQUET-1160 - [C++] Implement BYTE_ARRAY-backed Decimal readsPARQUET-1164 - [C++] Follow API changes in ARROW-1808PARQUET-1165 - [C++] Pin clang-format version to 4.0PARQUET-1166 - [API Proposal] Add GetRecordBatchReader in parquet/arrow/reader.hPARQUET-1177 - [C++] Add more extensive compiler warnings when using ClangPARQUET-1177 - [C++] Add more extensive compiler warnings when using ClangPARQUET-1196 - [C++] Provide a parquet_arrow example project incl. CMake setupPARQUET-1200 - [C++] Support reading a single Arrow column from a Parquet filePARQUET-1218 - [C++] More informative error message on too short pagesPARQUET-1225 - NaN values may lead to incorrect filtering under certain circumstancesPARQUET-1227 - Thrift crypto metadata structuresPARQUET-1256 - [C++] Add --print-key-value-metadata option to parquet_reader toolPARQUET-1256 - [C++] Add --print-key-value-metadata option to parquet_reader toolPARQUET-1267 - replace "unsafe" std::equal by std::memcmpPARQUET-1276 - [C++] Reduce the amount of memory used for writing null decimal valuesPARQUET-1279 - Use ASSERT_NO_FATAIL_FAILURE in C++ unit testsPARQUET-1301 - [C++] Crypto package in parquet-cppPARQUET-1308 - [C++] parquet::arrow should use thread pool, not ParallelForPARQUET-1323 - [C++] Fix compiler warnings with clang-6.0PARQUET-1332 - [C++] Add bloom filter utility classPARQUET-1340 - [C++] Fix Travis Ci valgrind errors related to std::random_devicePARQUET-1346 - [C++] Protect against null values data in empty Arrow arrayPARQUET-1348 - [C++] Allow Arrow FileWriter To Write FileMetaDataPARQUET-1350 - [C++] Use abstract ResizableBuffer instead of concrete PoolBufferPARQUET-1360 - [C++] Minor API + style changes follow up to PARQUET-1348PARQUET-1366 - [C++] Streamline use of Arrow bit-util.hPARQUET-1372 - [C++] Add an API to allow writing RowGroups based on their size rather than num_rowsPARQUET-1372 - [C++] Add an API to allow writing RowGroups based on their size rather than num_rowsPARQUET-1378 - [c++] Allow RowGroups with zero rows to be writtenPARQUET-1382 - [C++] Prepare for arrow::test namespace removalPARQUET-1392 - [C++] Supply row group indices to parquet::arrow::FileReader::ReadTablePARQUET-1398 - Separate iv_prefix for GCM and CTR modesPARQUET-1401 - RowGroup offset and total compressed size fieldsPARQUET-1427 - [C++] Move example executables and CLI tools to Apache Arrow repoPARQUET-1431 - [C++] Automaticaly set thrift to use boost for thrift versions before 0.11Bug Fixes
ARROW-1380 - [C++] Fix "still reachable" valgrind warnings when PLASMA_VALGRIND=1ARROW-1661 - [Python] Python 3.7 supportARROW-1799 - [Plasma C++] Make unittest does not create plasma store executableARROW-1996 - [Python] pyarrow.read_serialized cannot read concatenated recordsARROW-2027 - [C++] ipc::Message::SerializeTo does not pad the message bodyARROW-2220 - Change default fix version in merge tool to be the next mainline release versionARROW-2310 - Source release scripts fail with Java8ARROW-2646 - [C++/Python] Pandas roundtrip for date objectsARROW-2775 - [Python] ccache error when building manylinux1 wheelsARROW-2776 - [C++] Do not pass -Wno-noexcept-type for compilers that do not support itARROW-2782 - [Python] Ongoing Travis CI failures in Plasma unit testsARROW-2785 - [C++] Crash in json-integration-testARROW-2814 - [Python] Unify PyObject* sequence conversion paths for built-in sequences, NumPy arraysARROW-2854 - [C++/Python] Casting float NaN to int should raise an error on safe castARROW-2925 - [JS] Documentation failing in docker containerARROW-2965 - [Python] Possible uint64 overflow issues in python_to_arrow.ccARROW-2966 - [Python] Data type conversion errorARROW-2973 - [Python] pitrou/asv.git@customize_commands does not work with the "new" way of activating condaARROW-2974 - [Python] Replace usages of "source activate" with "conda activate" in CI scriptsARROW-2986 - [C++] /EHsc possibly needed for Visual Studio 2015 buildsARROW-2992 - [Python] Parquet benchmark failureARROW-2992 - [Python] Parquet benchmark failureARROW-3006 - [GLib] .gir/.typelib for GPU aren't installedARROW-3007 - [Packaging] libarrow-gpu10 deb for Ubuntu 18.04 has broken dependenciesARROW-3011 - [CI] Remove Slack notificationARROW-3012 - [Python] Installation crashes with setuptools_scm errorARROW-3013 - [Website] Fix download links on website for tarballs, checksumsARROW-3015 - [Python] Fix documentation typo for pa.uint8ARROW-3047 - [C++] cmake downloads and builds ORC even though it's installedARROW-3049 - [C++/Python] ORC reader fails on empty fileARROW-3053 - [Python] Pandas decimal conversion segfaultARROW-3056 - [Python] Indicate in NativeFile docstrings methods that are part of the RawIOBase API but not implementedARROW-3061 - [Java] headroom does not take into account reservationARROW-3065 - [Python] concat_tables() failing from bad Pandas MetadataARROW-3083 - [Python] Version in manylinux1 wheel builds is wrongARROW-3093 - [C++] Linking errors with ORC enabledARROW-3095 - [Python] test_plasma.py failsARROW-3098 - [Python] BufferReader doesn't adhere to the seek protocolARROW-3100 - [CI] C/glib build broken on OS XARROW-3125 - [Python] Update ASV instructionsARROW-3125 - [Python] Update ASV instructionsARROW-3132 - Regenerate 0.10.0 changelogARROW-3137 - [Python] pyarrow 0.10 requires newer version of numpy than specified in requirementsARROW-3140 - [Plasma] Plasma fails building with GPU enabledARROW-3141 - [Python] Tensorflow support in pyarrow wheels pins numpy>=1.14ARROW-3145 - [C++] Thrift compiler reruns in arrow/dbi/hiveserver2/thrift when using Ninja buildARROW-3173 - [Rust] dynamic_types example does not runARROW-3175 - [Java] Upgrade to official FlatBuffers release (Flatbuffers incompatibility)ARROW-3183 - [Python] get_library_dirs on Windows can give the wrong directoryARROW-3188 - [Python] Table.from_arrays segfaults if lists and schema are passedARROW-3190 - [C++] "WriteableFile" is misspelled, should be renamed "WritableFile" with deprecation for old nameARROW-3206 - [C++] Building with ARROW_HIVESERVER2=ON with unit tests disabled causes errorARROW-3227 - [Python] NativeFile.write shouldn't accept unicode stringsARROW-3228 - [Python] Immutability of bytes is ignoredARROW-3231 - [Python] Sphinx's autodoc_default_flags is now deprecatedARROW-3237 - [CI] Update linux packaging filenames in rat exclusion listARROW-3241 - [Plasma] test_plasma_list test failure on Ubuntu 14.04ARROW-3251 - [C++] Conversion warnings in cast.ccARROW-3256 - [JS] File footer and message metadata is inconsistentARROW-3271 - [Python] Manylinux1 builds timing out in Travis CIARROW-3279 - [C++] Allow linking Arrow tests dynamically on WindowsARROW-3299 - [C++] Appveyor builds failingARROW-3322 - [CI] Rust job always runs on AppVeyorARROW-3327 - [Python] manylinux container confusingARROW-3338 - [Python] Crash when schema and columns do not matchARROW-3342 - Appveyor builds have stopped triggering on GitHubARROW-3348 - Plasma store dies when an object that a dead client is waiting for gets created.ARROW-3354 - [Python] read_record_batch interfaces differ in pyarrow and pyarrow.cudaARROW-3369 - [Packaging] Wheel builds are failing due to wheel 0.32 releaseARROW-3370 - [Packaging] Centos 6 build is failingARROW-3373 - Fix bug in which plasma store can die when client gets multiple objects and object becomes available.ARROW-3374 - [Python] Dictionary has out-of-bound index when creating DictionaryArray from Pandas with NaNARROW-3390 - [C++] cmake file under windows msys2 system doesn't workARROW-3393 - [C++] Fix compiler warning in util/task-group-cc on clang 6ARROW-3394 - [Java] Remove duplicate dependency entry in FlightARROW-3403 - [Website] Source tarball link missing from install pageARROW-3420 - [C++] Fix outstanding include-what-you-use issues in src/arrow, src/parquet codebasesPARQUET-232 - minor compilation issuePARQUET-446 - Hide thrift dependency in parquet-cppPARQUET-454 - Address inconsistencies in boolean decodingPARQUET-455 - Fix compiler warnings on OS X / ClangPARQUET-457 - Add compressed data page unit testsPARQUET-469 - Roll back Thrift bindings to 0.9.0PARQUET-472 - Clean up InputStream ownership semantics in ColumnReaderPARQUET-505 - Column reader: automatically handle large data pagesPARQUET-507 - Improve runtime of rle-test.ccPARQUET-513 - Valgrind errors are not failing the Travis CI buildPARQUET-525 - Test coverage for malformed file failure modes on the read pathPARQUET-537 - LocalFileSource leaks resourcesPARQUET-549 - Add scanner and column reader tests for dictionary data pagesPARQUET-555 - Dictionary page metadata handling inconsistenciesPARQUET-561 - ParquetFileReader::Contents PIMPL missing a virtual destructorPARQUET-599 - ColumnWriter::RleEncodeLevels' size estimation might be wrongPARQUET-604 - Install writer.h headersPARQUET-614 - C++: Remove unneeded LZ4-related codePARQUET-620 - C++: Duplicate calls to ParquetFileWriter::Close cause duplicate metdata writesPARQUET-621 - C++: Uninitialised DecimalMetadata is readPARQUET-629 - RowGroupSerializer should only close itself oncePARQUET-639 - Do not export DCHECK in public headersPARQUET-643 - Add const modifier to schema pointer reference in ParquetFileWriterPARQUET-657 - [C++] Don't define DISALLOW_COPY_AND_ASSIGN if already definedPARQUET-658 - ColumnReader has no virtual destructorPARQUET-659 - [C++] Instantiated template visibility is broken on clang / OS XPARQUET-662 - [C++] ParquetException must be explicitly exported in dynamic librariesPARQUET-676 - MAX_VALUES_PER_LITERAL_RUN causes RLE encoding failurePARQUET-691 - [C++] Write ColumnChunk metadata after each column chunk in the filePARQUET-694 - C++: Revert default data page size back to 1MPARQUET-700 - C++: Disable dictionary encoding for boolean columnsPARQUET-701 - C++: Dictionary is written multiple times if close is called multiple times.PARQUET-702 - Add a writer + reader example with detailed commentsPARQUET-702 - Add a writer + reader example with detailed commentsPARQUET-703 - [C++] Validate num_values metadata for columns with nullsPARQUET-704 - [C++] scan-all.h is not being installedPARQUET-708 - [C++] RleEncoder does not account for "worst case scenario" in MaxBufferSize for bit_width > 1PARQUET-710 - Remove unneeded private member variables from RowGroupReader ABIPARQUET-711 - Use metadata builders in parquet writerPARQUET-711 - Use metadata builders in parquet writerPARQUET-718 - Reading boolean pages written by parquet-cpp failsPARQUET-719 - Fix WriterBatch API to handle NULL valuesPARQUET-720 - Parquet-cpp fails to link when included in multiple TUsPARQUET-739 - Rle-decoding uses static buffer that is shared accross threadsPARQUET-739 - Rle-decoding uses static buffer that is shared accross threadsPARQUET-741 - compression_buffer_ is reused although it shouldn'tPARQUET-742 - Add missing license headersPARQUET-745 - TypedRowGroupStatistics fails to PlainDecode min and max in ByteArrayTypePARQUET-747 - [C++] TypedRowGroupStatistics are not being exported in libparquet.soPARQUET-759 - Cannot store columns consisting of empty stringsPARQUET-760 - On switching from dictionary to the fallback encoding, an incorrect encoding is setPARQUET-764 - [CPP] Parquet Writer does not write Boolean values correctlyPARQUET-766 - C++: Expose ParquetFileReader through Arrow reader as constPARQUET-775 - C++: TrackingAllocator is not thread-safePARQUET-779 - Export TypedRowGroupStatistics in libparquetPARQUET-780 - WriterBatch API does not properly handle NULL values for byte array typesPARQUET-789 - [C++] Catch and translate ParquetException in parquet::arrow::FileReader::{ReadFlatColumn, ReadFlatTable}}PARQUET-793 - [CPP] Do not return incorrect statisticsPARQUET-797 - [C++] Update for API changes in ARROW-418PARQUET-799 - concurrent usage of the file reader APIPARQUET-812 - [C++] Failure reading BYTE_ARRAY data from file in parquet-compatibility projectPARQUET-816 - [C++] Failure decoding sample dict-encoded file from parquet-compatibility projectPARQUET-818 - [C++] Refactor library to share IO, Buffer, and memory management abstractions with Apache ArrowPARQUET-818 - [C++] Refactor library to share IO, Buffer, and memory management abstractions with Apache ArrowPARQUET-819 - C++: Trying to install non-existing parquet/arrow/utils.hPARQUET-827 - [C++] Incorporate addition of arrow::MemoryPool::ReallocatePARQUET-828 - [C++] "version" field set improperly in file metadataPARQUET-837 - [C++] SerializedFile::ParseMetaData uses Seek, followed by Read, and could have race conditionsPARQUET-841 - [C++] Writing wrong format version when using ParquetVersion::PARQUET_1_0PARQUET-842 - [C++] Impala rejects DOUBLE columns if decimal metadata is setPARQUET-843 - [C++] Impala unable to read files created by parquet-cppPARQUET-846 - [CPP] CpuInfo::Init() is not thread safePARQUET-880 - [CPP] Prevent destructors from throwingPARQUET-888 - C++ Memory leak in RowGroupSerializerPARQUET-889 - Fix compilation when PARQUET_USE_SSE is onPARQUET-892 - [C++] Clean up link library targets in CMake filesPARQUET-895 - Reading of nested columns is brokenPARQUET-898 - [C++] Change Travis CI OS X image to Xcode 6.4 and fix our thirdparty buildPARQUET-908 - Fix for PARQUET-890 introduces undefined symbol in libparquet_arrow.soPARQUET-914 - [C++] Throw more informative exception when user writes too many values to a column in a row groupPARQUET-915 - Support Arrow Time Types in SchemaPARQUET-918 - FromParquetSchema API crashes on nested schemasPARQUET-918 - FromParquetSchema API crashes on nested schemasPARQUET-919 - [C++] Account for API changes in ARROW-683PARQUET-923 - [C++] Account for Time metadata changes in ARROW-686PARQUET-933 - [C++] Account for Arrow Table API changes coming in ARROW-728PARQUET-936 - [C++] parquet::arrow::WriteTable can enter infinite loop if chunk_size is 0PARQUET-943 - [C++] Overflow build error on x86PARQUET-947 - [C++] Refactor to account for ARROW-795 Arrow core library consolidationPARQUET-958 - [C++] Print Parquet metadata in JSON formatPARQUET-958 - [C++] Print Parquet metadata in JSON formatPARQUET-963 - [C++] Disallow reading struct types in Arrow reader for nowPARQUET-965 - [C++] FIXED_LEN_BYTE_ARRAY types are unhandled in the Arrow readerPARQUET-979 - [C++] Limit size of min, max or disable stats for long binary typesPARQUET-992 - [C++] parquet/compression.h leaks zlib.hPARQUET-995 - [C++] Int96 reader in parquet_arrow uses size of Int96Type instead of Int96PARQUET-997 - Fix override compiler warningsPARQUET-1002 - [C++] Compute statistics based on Logical TypesPARQUET-1003 - [C++] Modify DEFAULT_CREATED_BY value for every new release versionPARQUET-1007 - [C++ ] Update parquet.thrift from https://github.com/apache/parquet-format PARQUET-1029 - [C++] TypedColumnReader/TypeColumnWriter symbols are no longer being exportedPARQUET-1029 - [C++] TypedColumnReader/TypeColumnWriter symbols are no longer being exportedPARQUET-1033 - Mismatched Read and WritePARQUET-1038 - Key value metadata should be nullptr if not setPARQUET-1040 - Missing writer method implementationsPARQUET-1042 - C++: Compilation breaks on GCC 4.8PARQUET-1048 - [C++] Static linking of libarrow is no longer supportedPARQUET-1048 - [C++] Static linking of libarrow is no longer supportedPARQUET-1054 - [C++] Account for Arrow API changes in ARROW-1199PARQUET-1071 - [C++] parquet::arrow::FileWriter::Close is not idempotentPARQUET-1085 - [C++] Backwards compatibility from macro cleanup in transitive dependencies in ARROW-1452PARQUET-1088 - [CPP] remove parquet_version.h from version control since it gets auto generatedPARQUET-1090 - [C++] Fix int32 overflow in Arrow table writer, add max row group size propertyPARQUET-1098 - [C++] Install new header in parquet/utilPARQUET-1100 - [C++] Reading repeated types should decode number of records rather than number of valuesPARQUET-1108 - [C++] Fix Int96 comparatorsPARQUET-1114 - Apply fix for ARROW-1601 and ARROW-1611 to parquet-cppPARQUET-1121 - C++: DictionaryArrays of NullType cannot be writtenPARQUET-1123 - [C++] Update parquet-cpp to use Arrow's AssertArraysEqualPARQUET-1138 - [C++] Fix compilation with Arrow 0.7.1PARQUET-1167 - [C++] FieldToNode function should return a status when throwing an exceptionPARQUET-1175 - [C++] Fix usage of deprecated Arrow APIPARQUET-1179 - [C++] Support Apache Thrift 0.11PARQUET-1180 - C++: Fix behaviour of num_children element of primitive nodesPARQUET-1193 - [CPP] Implement ColumnOrder to support min_value and max_valuePARQUET-1226 - [C++] Fix new build warnings with clang 5.0PARQUET-1233 - [CPP ]Enable option to switch between stl classes and boost classes for thrift headerPARQUET-1245 - [C++] Segfault when writing Arrow table with duplicate columnsPARQUET-1255 - [C++] Exceptions thrown in some testsPARQUET-1265 - Segfault on static ApplicationVersion initializationPARQUET-1268 - [C++] Conversion of Arrow null list columns failsPARQUET-1270 - [C++] Executable tools do not get installedPARQUET-1272 - [C++] ScanFileContents reports wrong row count for nested columnsPARQUET-1273 - [Python] Error writing to partitioned Parquet datasetPARQUET-1274 - [Python] SegFault in pyarrow.parquet.write_table with specific optionsPARQUET-1283 - [C++] FormatStatValue appends trailing space to string and int96PARQUET-1307 - [C++] memory-test fails with latest ArrowPARQUET-1315 - [C++] ColumnChunkMetaData.has_dictionary_page() should return bool, not int64_tPARQUET-1333 - [C++] Reading of files with dictionary size 0 fails on Windows with bad_allocPARQUET-1334 - [C++] memory_map parameter seems missleading in parquet file openerPARQUET-1357 - [C++] FormatStatValue truncates binary statistics on zero characterPARQUET-1358 - [C++] index_page_offset should be unset as it is not supported.PARQUET-1369 - [Python] Unavailable Parquet column statistics from Spark-generated filePARQUET-1384 - [C++] Clang compiler warnings in bloom_filter-test.cc
Apache Arrow 0.10.0 (2018-08-06)
Bug Fixes
ARROW-198 - [Java] OutOfMemoryError for vector test caseARROW-640 - [Python] Arrow scalar values should have a sensible __hash__ and comparisonARROW-2020 - [Python] Parquet segfaults if coercing ns timestamps and writing 96-bit timestampsARROW-2059 - [Python] Possible performance regression in Feather read/write pathARROW-2101 - [Python] from_pandas reads 'str' type as binary Arrow data with Python 2ARROW-2122 - [Python] Pyarrow fails to serialize dataframe with timestamp.ARROW-2182 - [Python] ASV benchmark setup does not account for C++ library changingARROW-2189 - [C++] Seg. fault on make_shared
ARROW-2193 - [Plasma] plasma_store has runtime dependency on Boost shared libraries when ARROW_BOOST_USE_SHARED=onARROW-2195 - [Plasma] Segfault when retrieving RecordBatch from plasma storeARROW-2247 - [Python] Statically-linking boost_regex in both libarrow and libparquet results in segfaultARROW-2273 - Cannot deserialize pandas SparseDataFrameARROW-2300 - [Python] python/testing/test_hdfs.sh no longer worksARROW-2305 - [Python] Cython 0.25.2 compilation failureARROW-2314 - [Python] Union array slicing is defectiveARROW-2326 - [Python] cannot import pip installed pyarrow on OS X (10.9)ARROW-2328 - Writing a slice with feather ignores the offsetARROW-2331 - [Python] Fix indexing implementationsARROW-2333 - [Python] boost bundling fails in setup.pyARROW-2342 - [Python] Aware timestamp type fails picklingARROW-2346 - [Python] PYARROW_CXXFLAGS doesn't accept multiple optionsARROW-2349 - [Python] Boost shared library bundling is broken for MSVCARROW-2351 - [C++] StringBuilder::append(vector...) not implemented
ARROW-2354 - [C++] PyDecimal_Check() is much too slowARROW-2355 - [Python] Unable to import pyarrow [0.9.0] OSXARROW-2357 - Benchmark PandasObjectIsNullARROW-2368 - DecimalVector#setBigEndian is not padding correctly for negative valuesARROW-2369 - Large (>~20 GB) files written to Parquet via PyArrow are corruptedARROW-2370 - [GLib] include path is wrong on Meson buildARROW-2371 - [GLib] gio-2.0 isn't required on GNU Autotools buildARROW-2372 - [Python] ArrowIOError: Invalid argument when reading Parquet fileARROW-2375 - [Rust] Buffer should release memory when droppedARROW-2377 - [GLib] Travis-CI failuresARROW-2380 - [Python] Correct issues in numpy_to_arrow conversion routinesARROW-2382 - [Rust] List was not using memory safely
ARROW-2383 - [C++] Debian packages need to depend on libprotobufARROW-2387 - [Python] negative decimal values get spurious rescaling errorARROW-2391 - [Python] Segmentation fault from PyArrow when mapping Pandas datetime column to pyarrow.date64ARROW-2393 - [C++] arrow/status.h does not define ARROW_CHECK needed for ARROW_CHECK_OKARROW-2403 - [C++] arrow::CpuInfo::model_name_ destructed twice on exitARROW-2405 - [C++] is missing in plasma/client.h
ARROW-2418 - [Rust] List builder fails due to memory not being reserved correctlyARROW-2419 - [Site] Website generation depends on local timezoneARROW-2420 - [Rust] Memory is never releasedARROW-2421 - [C++] Update LLVM version in cpp READMEARROW-2423 - [Python] PyArrow datatypes raise ValueError on equality checks against non-PyArrow objectsARROW-2424 - [Rust] Missing import causing broken buildARROW-2425 - [Rust] Array::from missing mapping for u8 typeARROW-2426 - [CI] glib build failureARROW-2432 - [Python] from_pandas fails when converting decimals if have None valuesARROW-2437 - [C++] Change of arrow::ipc::ReadMessage signature breaks ABI compabilityARROW-2438 - [Rust] memory_pool.rs misses license headerARROW-2441 - [Rust] Builder::slice_mut assertions are too strict
ARROW-2443 - [Python] Conversion from pandas of empty categorical fails with ArrowInvalidARROW-2450 - [Python] Saving to parquet fails for empty listsARROW-2452 - [TEST] Spark integration test fails with permission errorARROW-2454 - [Python] Empty chunked array slice crashesARROW-2455 - [C++] The bytes_allocated_ in CudaContextImpl isn't initializedARROW-2457 - garrow_array_builder_append_values() won't work for large arraysARROW-2459 - pyarrow: Segfault with pyarrow.deserialize_pandasARROW-2462 - [C++] Segfault when writing a parquet table containing a dictionary column from Record Batch StreamARROW-2465 - [Plasma] plasma_store fails to find libarrow_gpu.soARROW-2466 - [C++] misleading "append" flag to FileOutputStreamARROW-2468 - [Rust] Builder::slice_mut should take mut selfARROW-2471 - [Rust] Assertion when pushing value to Builder/ListBuilder with zero capacityARROW-2473 - [Rust] List assertion error with list of zero lengthARROW-2474 - [Rust] Add windows support for memory pool abstractionARROW-2489 - [Plasma] test_plasma.py crashesARROW-2491 - [Python] Array.from_buffers does not work for ListArrayARROW-2492 - [Python] Prevent segfault on accidental call of pyarrow.ArrayARROW-2500 - [Java] IPC Writers/readers are not always setting validity bits correctlyARROW-2502 - [Rust] Restore Windows CompatibilityARROW-2503 - [Python] Trailing space character in RowGroup statistics of pyarrow.parquet.ParquetFileARROW-2509 - [CI] Intermittent npm failuresARROW-2510 - [Python] Segmentation fault when converting empty column as categoricalARROW-2511 - BaseVariableWidthVector.allocateNew is not throwing OOM when it can't allocate memoryARROW-2514 - [Python] Inferring / converting nested Numpy array is very slowARROW-2515 - Errors with DictionaryArray inside of ListArray or other DictionaryArrayARROW-2518 - [Java] Restore Java unit tests and javadoc test to CI matrixARROW-2530 - [GLib] Out-of-source build is failedARROW-2534 - [C++] libarrow.so leaks zlib symbolsARROW-2545 - [Python] Arrow fails linking against statically-compiled PythonARROW-2554 - pa.array type inference bug when using NS-timestampARROW-2557 - [Rust] Add badge for code coverage in READMEARROW-2561 - [C++] Crash in cuda-test shutdown with coverage enabledARROW-2564 - [C++] Rowwise Tutorial is out of dateARROW-2565 - [Plasma] new subscriber cannot receive notifications about existing objectsARROW-2570 - [Python] Add support for writing parquet files with LZ4 compressionARROW-2571 - [C++] Lz4Codec doesn't properly handle empty dataARROW-2575 - [Python] Exclude hidden files when reading Parquet datasetARROW-2578 - [Plasma] Valgrind errors related to std::random_deviceARROW-2589 - [Python] test_parquet.py regression with Pandas 0.23.0ARROW-2593 - [Python] TypeError: data type "mixed-integer" not understoodARROW-2594 - [Java] Vector reallocation does not properly clear reused buffersARROW-2599 - [Python] pip install is not working without Arrow C++ being installedARROW-2601 - [Python] MemoryPool bytes_allocated causes segARROW-2603 - [Python] from pandas raises ArrowInvalid for date(time) subclassesARROW-2615 - [Rust] Refactor introduced a bug around Arrays of StringARROW-2622 - [C++] Array methods IsNull and IsValid are not complementaryARROW-2629 - [Plasma] Iterator invalidation for pending_notifications_ARROW-2630 - [Java] Typo in the documentARROW-2632 - [Java] ArrowStreamWriter accumulates ArrowBlock but does not use themARROW-2640 - JS Writer should serialize schema metadataARROW-2642 - [Python] Fail building parquet binding on WindowsARROW-2643 - [C++] Travis-CI build failure with cpp toolchain enabledARROW-2644 - [Python] parquet binding fails building on AppVeyorARROW-2655 - [C++] Failure with -Werror=conversion on gcc 7.3.0ARROW-2657 - Segfault when importing TensorFlow after PyarrowARROW-2668 - [C++] -Wnull-pointer-arithmetic warning with dlmalloc.c on clang 6.0, Ubuntu 14.04ARROW-2669 - [C++] EP_CXX_FLAGS not passed on when building gbenchmarkARROW-2675 - Arrow build error with clang-10 (Apple Clang / LLVM)ARROW-2683 - [Python] Resource Warning (Unclosed File) when using pyarrow.parquet.read_table()ARROW-2690 - [C++] Plasma does not follow style conventions for variable and function namesARROW-2691 - [Rust] Travis fails due to formatting diffARROW-2693 - [Python] pa.chunked_array causes a segmentation fault on empty inputARROW-2694 - [Python] ArrayValue string conversion returns the representation instead of the converted python object stringARROW-2698 - [Python] Exception when passing a string to Table.columnARROW-2711 - [Python/C++] Pandas-Arrow doesn't roundtrip when column of lists has empty first elementARROW-2715 - Address apt flakiness with launchpad.netARROW-2716 - [Python] Make manylinux1 base image independent of Python patch releasesARROW-2721 - [C++] Link error with Arrow C++ build with -DARROW_ORC=ON on CentOS 7ARROW-2722 - [Python] ndarray to arrow conversion fails when downcasted from pandas to_numericARROW-2723 - [C++] arrow-orc.pc is missingARROW-2726 - [C++] The latest Boost version is wrongARROW-2727 - [Java] Unable to build java/adapters moduleARROW-2741 - [Python] pa.array from np.datetime[D] and type=pa.date64 produces invalid resultsARROW-2744 - [Python] Writing to parquet crashes when writing a ListArray of empty listsARROW-2745 - [C++] ORC ExternalProject needs to declare dependency on vendored protobufARROW-2747 - [CI] [Plasma] huge tables test failure on TravisARROW-2754 - [Python] When installing pyarrow via pip, a debug build is createdARROW-2770 - [Packaging] Account for conda-forge compiler migration in conda recipesARROW-2773 - [Python] Corrected parquet docs partition_cols parameter nameARROW-2781 - [Python] Download boost using curl in manylinux1 imageARROW-2787 - [Python] Memory Issue passing table from python to c++ via cythonARROW-2795 - [Python] Run TensorFlow import workaround only on LinuxARROW-2806 - [Python] Inconsistent handling of np.nanARROW-2810 - [Plasma] Plasma public headers leak flatbuffers.hARROW-2812 - [Ruby] StructArray#[] raises NoMethodErrorARROW-2820 - [Python] RecordBatch.from_arrays does not validate array lengths are all equalARROW-2823 - [C++] Search for flatbuffers in /lib64
ARROW-2841 - [Go] Fix recent Go build failures in Travis CIARROW-2850 - [C++/Python] PARQUET_RPATH_ORIGIN=ON missing in manylinux1 buildARROW-2851 - [C++] Update RAT excludes for new install file namesARROW-2852 - [Rust] Mark Array as Sync and SendARROW-2856 - [Python/C++] Array constructor should not truncate floats when casting to intARROW-2862 - [C++] Ensure thirdparty download directory has been created in thirdparty/download_thirdparty.shARROW-2867 - [Python] Incorrect example for Cython usageARROW-2871 - [Python] Array.to_numpy is invalid for boolean arraysARROW-2872 - [Python] Add pytest mark to opt into TensorFlow-related unit testsARROW-2876 - [Packaging] Crossbow builds can hang if you cloned using SSHARROW-2877 - [Packaging] crossbow submit results in duplicate Travis CI buildARROW-2878 - [Packaging] README.md does not mention setting GitHub API token in user's crossbow repo settingsARROW-2883 - [Plasma] Compilation warningsARROW-2891 - [Python] Preserve schema in write_to_datasetARROW-2894 - [Glib] Format tests broken due to recent refactorARROW-2895 - [Ruby] CI isn't ran when C++ is changedARROW-2896 - [GLib] export are missingARROW-2901 - [Java] Build is failing on Java9ARROW-2902 - [Python] HDFS Docker integration tests leave around files created by rootARROW-2903 - [C++] Setting -DARROW_HDFS=OFF breaks arrow build when linking against boost librariesARROW-2911 - [Python] Parquet binary statistics that end in '\0' truncate last byteARROW-2917 - [Python] Tensor requiring gradiant cannot be serialized with pyarrow.serializeARROW-2920 - [Python] Segfault with pytorch 0.4ARROW-2926 - [Python] ParquetWriter segfaults in example where passed schema and table schema do not matchARROW-2930 - [C++] Trying to set target properties on not existing CMake targetARROW-2940 - [Python] Import error with pytorch 0.3ARROW-2945 - [Packaging] Update argument check for 02-source.shARROW-2955 - [Python] Typo in pyarrow's HDFS API resultARROW-2963 - [Python] Deadlock during fork-join and use_threads=TrueARROW-2978 - [Rust] Travis CI build is failingARROW-2982 - The "--show-progress" option is only supported in wget 1.16 and higherARROW-3210 - [Python] Creating ParquetDataset creates partitioned ParquetFiles with mismatched Parquet schemasNew Features and Improvements
ARROW-530 - C++/Python: Provide subpools for better memory allocation trackingARROW-564 - [Python] Add methods to return vanilla NumPy arrays (plus boolean mask array if there are nulls)ARROW-665 - C++: Move zeroing logic for (re)allocations to the AllocatorARROW-889 - [C++] Implement arrow::PrettyPrint for ChunkedArrayARROW-902 - [C++] Build C++ project including thirdparty dependencies from local tarballsARROW-906 - [C++] Serialize Field metadata to IPC metadataARROW-1018 - [C++] Add option to create FileOutputStream, ReadableFile from OS file descriptorARROW-1163 - [Plasma][Java] Java client for PlasmaARROW-1388 - [Python] Add Table.drop method for removing columnsARROW-1454 - [Python] More informative error message when attempting to write an unsupported Arrow type to Parquet formatARROW-1715 - [Python] Implement pickling for Column, ChunkedArray, RecordBatch, TableARROW-1722 - [C++] Add linting script to look for C++/CLI issuesARROW-1731 - [Python] Provide for selecting a subset of columns to convert in RecordBatch/Table.from_pandasARROW-1744 - [Plasma] Provide TensorFlow operator to read tensors from plasmaARROW-1780 - [Java] JDBC Adapter for Apache ArrowARROW-1858 - [Python] Add documentation about parquet.write_to_dataset and related methodsARROW-1868 - [Java] Change vector getMinorType to use MinorType instead of Types.MinorTypeARROW-1886 - [Python] Add function to "flatten" structs within tablesARROW-1913 - [Java] Fix Javadoc generation bugs with JDK8ARROW-1928 - [C++] Add benchmarks comparing performance of internal::BitmapReader/Writer with naive approachesARROW-1954 - [Python] Add metadata accessor to pyarrow.FieldARROW-1964 - [Python] Expose Builder classesARROW-2014 - [Python] Document read_pandas method in pyarrow.parquetARROW-2055 - [Java] Upgrade to Java 8ARROW-2060 - [Python] Documentation for creating StructArray using from_arrays or a sequence of dictsARROW-2061 - [C++] Run ASAN builds in Travis CIARROW-2074 - [Python] Allow type inference for struct arraysARROW-2097 - [Python] Suppress valgrind stdout/stderr in Travis CI builds when there are no errorsARROW-2100 - [Python] Drop Python 3.4 supportARROW-2140 - [Python] Conversion from Numpy float16 array unimplementedARROW-2141 - [Python] Conversion from Numpy object array to varsize binary unimplementedARROW-2147 - [Python] Type inference doesn't work on lists of Numpy arraysARROW-2207 - [GLib] Support decimal typeARROW-2222 - [C++] Add option to validate Flatbuffers messagesARROW-2224 - [C++] Get rid of boost regex usageARROW-2241 - [Python] Simple script for running all current ASV benchmarks at a commit or tagARROW-2264 - [Python] Efficiently serialize numpy arrays with dtype of unicode fixed length stringARROW-2267 - Rust bindingsARROW-2276 - [Python] Tensor could implement the buffer protocolARROW-2281 - [Python] Expose MakeArray to construct arrays from buffersARROW-2285 - [Python] Can't convert Numpy string arraysARROW-2286 - [Python] Allow subscripting pyarrow.lib.StructValueARROW-2287 - [Python] chunked array not iterable, not indexableARROW-2299 - [Go] Go language implementationARROW-2301 - [Python] Add source distribution publishing instructions to package / release management documentationARROW-2302 - [GLib] Run autotools and meson Linux builds in same Travis CI build entryARROW-2308 - Serialized tensor data should be 64-byte aligned.ARROW-2315 - [C++/Python] Add method to flatten a struct arrayARROW-2319 - [C++] Add buffered output class implementing OutputStream interfaceARROW-2322 - Document requirements to run dev/release/01-perform.shARROW-2325 - [Python] Update setup.py to use Markdown project descriptionARROW-2330 - [C++] Optimize delta buffer creation with partially finishable array buildersARROW-2332 - [Python] Provide API for reading multiple Feather filesARROW-2332 - [Python] Provide API for reading multiple Feather filesARROW-2334 - [C++] Update boost to 1.66.0ARROW-2335 - [Go] Move Go README one directory higherARROW-2340 - [Website] Add blog post about Go codebase donationARROW-2341 - [Python] pa.union() mode argument unintuitiveARROW-2343 - [Java/Packaging] Run mvn clean in API doc buildsARROW-2344 - [Go] Run Go unit tests in Travis CIARROW-2345 - [Documentation] Fix bundle exec and set sphinx nosidebar to TrueARROW-2348 - [GLib] Remove Go exampleARROW-2350 - Shrink size of spark_integration Docker containerARROW-2353 - Test correctness of built wheel on AppVeyorARROW-2361 - [Rust] Start native Rust ImplementationARROW-2364 - [Plasma] PlasmaClient::Get() could take vector of object idsARROW-2376 - [Rust] Travis should run tests for Rust libraryARROW-2378 - [Rust] Use rustfmt to format source codeARROW-2381 - [Rust] Buffer should have an Iterator
ARROW-2384 - Rust: Use Traits rather than defining methods directlyARROW-2385 - [Rust] Implement to_json() for Field and DataTypeARROW-2388 - [C++] Arrow::StringBuilder::Append() uses null_bytes not valid_bytesARROW-2389 - [C++] Add StatusCode::OverflowErrorARROW-2390 - [C++/Python] CheckPyError() could inspect exception typeARROW-2394 - [Python] Correct flake8 errors in benchmarksARROW-2395 - [Python] Correct flake8 errors outside of pyarrow/ directoryARROW-2396 - Unify Rust ErrorsARROW-2397 - Document changes in Tensor encoding in IPC.md.ARROW-2398 - [Rust] Provide a zero-copy builder for type-safe Buffer
ARROW-2400 - [C++] Status destructor is expensiveARROW-2401 - Support filters on Hive partitioned Parquet filesARROW-2402 - [C++] FixedSizeBinaryBuilder::Append lacks "const char*" overloadARROW-2404 - Fix declaration of 'type_id' hides class member warning in msvc buildARROW-2407 - [GLib] Add garrow_string_array_builder_append_values()ARROW-2408 - [Rust] It should be possible to get a &mut[T] from Builder
ARROW-2408 - [Rust] It should be possible to get a &mut[T] from Builder
ARROW-2411 - [C++] Add method to append batches of null-terminated strings to StringBuilderARROW-2413 - [Rust] Remove useless use of `format!`ARROW-2414 - [Documentation] Fix miscellaneous documentation typosARROW-2415 - [Rust] Fix using references in pattern matchingARROW-2416 - [C++] Support system libprotobufARROW-2417 - [Rust] Review APIs for safetyARROW-2422 - [Python] Support more filter operators on Hive partitioned Parquet filesARROW-2427 - [C++] ReadAt implementations suboptimalARROW-2430 - MVP for branch based packaging automationARROW-2433 - [Rust] Add Builder.push_slice(&[T])ARROW-2434 - [Rust] Add windows supportARROW-2435 - [Rust] Add memory pool abstraction.ARROW-2436 - [Rust] Add windows CIARROW-2439 - [Rust] Run license header checks also in Rust CI entryARROW-2440 - [Rust] Implement ListBuilder
ARROW-2442 - [C++] Disambiguate Builder::Append overloadsARROW-2445 - [Rust] Add documentation and make some fields privateARROW-2448 - Segfault when plasma client goes out of scope before buffer.ARROW-2451 - Handle more dtypes efficiently in custom numpy array serializer.ARROW-2453 - [Python] Improve Table column accessARROW-2458 - [Plasma] PlasmaClient uses global variableARROW-2463 - [C++] Update flatbuffers to 1.9.0ARROW-2464 - [Python] Use a python_version marker instead of a conditionARROW-2469 - Make out arguments last in ReadMessage API.ARROW-2470 - [C++] FileGetSize() should not seekARROW-2472 - [Rust] The Schema and Fields types should not have public attributesARROW-2477 - [Rust] Set up code coverage in CIARROW-2478 - [C++] Introduce a checked_cast function that performs a dynamic_cast in debug modeARROW-2479 - [C++] Have a global thread poolARROW-2480 - [C++] Enable casting the value of a decimal to int32_t or int64_tARROW-2481 - [Rust] Move calls to free() into memory.rsARROW-2482 - [Rust] support nested typesARROW-2484 - [C++] Document ABI compliance checkingARROW-2485 - [C++] Output diff when run_clang_format.py reports a changeARROW-2486 - [C++/Python] Provide a Docker image that contains all dependencies for developmentARROW-2488 - [C++] List Boost 1.67 as supported versionARROW-2493 - [Python] Add support for pickling to buffers and arraysARROW-2494 - Return status codes from PlasmaClient::SealARROW-2498 - [Java] Upgrade to JDK 1.8ARROW-2499 - [C++] Add iterator facility for Python sequencesARROW-2505 - [C++] Disable MSVC warning C4800ARROW-2506 - [Plasma] Build error on macOSARROW-2507 - [Rust] Don't take a reference when not neededARROW-2508 - [Python] pytest API changes make tests failARROW-2513 - [Python] DictionaryType should give access to index type and dictionary arrayARROW-2516 - AppVeyor Build Matrix should be specific to the changes made in a PRARROW-2521 - [Rust] Refactor Rust API to use traits and genericsARROW-2522 - [C++] Version shared library filesARROW-2525 - [GLib] Add garrow_struct_array_flatten()ARROW-2526 - [GLib] Update .gitignoreARROW-2527 - [GLib] Enable GPU documentARROW-2528 - [Rust] Add trait bounds for T in Buffer/ListARROW-2529 - [C++] Update mention of clang-format to 5.0 in the docsARROW-2531 - [C++] Update clang bits to 6.0ARROW-2533 - [CI] Fast finish failing AppVeyor buildsARROW-2536 - [Rust] ListBuilder uses wrong initial size for offset builderARROW-2537 - [Ruby] ImportARROW-2539 - [Plasma] Use unique_ptr instead of raw pointerARROW-2540 - [Plasma] add constructor/destructor to make sure dlfree is called automaticallyARROW-2541 - [Plasma] Clean up macro usageARROW-2543 - [Rust] CI should cache dependencies for faster buildsARROW-2544 - [CI] Run C++ tests with two jobs on Travis-CIARROW-2547 - [Format] Fix off-by-one in List> example
ARROW-2548 - [Format] Clarify `List` Array example
ARROW-2549 - [GLib] Apply arrow::StatusCodes changes to GArrowErrorARROW-2550 - [C++] Add missing status codes into arrow::StatusCode::CodeAsString()ARROW-2551 - [Plasma] Improve notification logicARROW-2552 - [Plasma] Unit tests are flakyARROW-2553 - [Python] Set MACOSX_DEPLOYMENT_TARGET in wheel buildARROW-2558 - [Plasma] avoid walk through all the objects when a client disconnectsARROW-2562 - [C++] Upload coverage data to codecov.ioARROW-2563 - [Rust] Poor caching in Travis-CIARROW-2566 - [CI] Add codecov.io badge to READMEARROW-2567 - [C++/Python] Unit is ignored on comparison of TimestampArraysARROW-2568 - [Python] Expose thread pool size setting to Python, and deprecate "nthreads"ARROW-2569 - [C++] Improve thread pool size heuristicARROW-2574 - [CI] Collect and publish Python coverageARROW-2576 - [GLib] Add abs functions for Decimal128.ARROW-2577 - [Plasma] Add ASV benchmarksARROW-2580 - [GLib] Fix abs functions for Decimal128ARROW-2582 - [GLib] Add negate functions for Decimal128ARROW-2585 - [C++] Add Decimal128::FromBigEndianARROW-2586 - [C++] Make child builders of ListBuilder and StructBuilder shared_ptr'sARROW-2595 - [Plasma] operator[] creates entries in mapARROW-2596 - [GLib] Use the default value of GTK-DocARROW-2597 - [Plasma] remove UniqueIDHasherARROW-2604 - [Java] Add method overload for VarCharVector.set(int,String)ARROW-2608 - [Java/Python] Add pyarrow.{Array,Field}.from_jvm / jvm_bufferARROW-2611 - [Python] Python 2 integer serializationARROW-2612 - [Plasma] Fix deprecated PLASMA_DEFAULT_RELEASE_DELAYARROW-2613 - [Docs] Update the gen_apidocs docker scriptARROW-2614 - [CI] Remove 'group: deprecated' in TravisARROW-2626 - [Python] pandas ArrowInvalid message should include failing column nameARROW-2634 - [Go] Add LICENSE additions for Go subprojectARROW-2635 - [Ruby] LICENSE.txt isn't suitableARROW-2636 - [Ruby] "Unofficial" package note is missingARROW-2638 - [Python] Prevent calling extension class constructors directlyARROW-2639 - [Python] Remove unnecessary _check_nullptr methodsARROW-2641 - [C++] Investigate spurious memset() callsARROW-2645 - [Java] ArrowStreamWriter accumulates DictionaryBatch ArrowBlocksARROW-2649 - [C++] Add std::generate()-like function for faster bitmap writingARROW-2656 - [Python] Improve ParquetManifest creation timeARROW-2660 - [Python] Experiment with zero-copy picklingARROW-2661 - [Python/C++] Allow passing HDFS Config values via map/dict instead of needing an hdfs-site.xml fileARROW-2662 - [Python] Add to_pandas / to_numpy to ChunkedArrayARROW-2663 - [Python] Make dictionary_encode and unique accesible on Column / ChunkedArrayARROW-2664 - [Python] Implement __getitem__ / slicing on BufferARROW-2666 - [Python] numpy.asarray should trigger to_pandas on Array/ChunkedArrayARROW-2672 - [Python] Build ORC extension in manylinux1 wheelsARROW-2674 - [Packaging] Start building nightliesARROW-2676 - [Packaging] Deploy build artifacts to github releasesARROW-2677 - [Python] Expose Parquet ZSTD compressionARROW-2678 - [GLib] Add extra information to common build problems on macOSARROW-2680 - [Python] Add documentation about type inference in Table.from_pandasARROW-2682 - [CI] Notify in Slack about broken buildsARROW-2689 - [Python] Remove references to timestamps_to_ms argument from documentationARROW-2692 - [Python] Add test for writing dictionary encoded columns to chunked Parquet filesARROW-2695 - [Python] Prevent calling scalar contructors directlyARROW-2696 - [JAVA] enhance AllocationListener with an onFailedAllocation() callARROW-2699 - [C++/Python] Add Table method that replaces a column with a new supplied columnARROW-2700 - [Python] Add simple examples to Array.cast docstringARROW-2701 - [C++] Make MemoryMappedFile resizableARROW-2704 - [Java] IPC stream handling should be more friendly to low level processingARROW-2713 - [Packaging] Fix linux package buildsARROW-2717 - [Packaging] Postfix conda artifacts with target archARROW-2718 - [Packaging] GPG sign downloaded artifactsARROW-2724 - [Packaging] Determine whether all the expected artifacts are uploadedARROW-2725 - [JAVA] make Accountant.AllocationOutcome publicly visibleARROW-2729 - [GLib] Add decimal128 array builderARROW-2731 - Allow usage of external ORC libraryARROW-2732 - Update brew packages for macOSARROW-2733 - [GLib] Cast garrow_decimal128 to gint64ARROW-2738 - [GLib] Use Brewfile on installation processARROW-2739 - [GLib] Use G_DECLARE_DERIVABLE_TYPE for GArrowDecimalDataType and GArrowDecimal128ArrayBuilderARROW-2740 - [Python] Add address property to BufferARROW-2742 - [Python] Allow Table.from_batches to use Iterator of ArrowRecordBatchesARROW-2748 - [GLib] Add garrow_decimal_data_type_get_scale() (and _precision())ARROW-2749 - [GLib] Rename *garrow_decimal128_array_get_value to *garrow_decimal128_array_format_valueARROW-2751 - [GLib] Add garrow_table_replace_column()ARROW-2752 - [GLib] Document garrow_decimal_data_type_new()ARROW-2753 - [GLib] Add garrow_schema_*_field()ARROW-2755 - [Python] Allow using Ninja to build extensionARROW-2756 - [Python] Remove redundant imports and minor fixes in parquet testsARROW-2758 - [Plasma] Use Scope enum in PlasmaARROW-2760 - [Python] Remove legacy property definition syntax from parquet module and test themARROW-2761 - Support set filter operators on Hive partitioned Parquet filesARROW-2763 - [Python] Make parquet _metadata file accessible from ParquetDatasetARROW-2780 - [Go] Run code coverage analysisARROW-2784 - [C++] MemoryMappedFile::WriteAt allow writing past the endARROW-2790 - [C++] Buffers contain uninitialized memoryARROW-2790 - [C++] Buffers contain uninitialized memoryARROW-2791 - [Packaging] Build Ubuntu 18.04 packagesARROW-2792 - [Packaging] Consider uploading tarballs to avoid naming conflictsARROW-2794 - [Plasma] Add Delete method for multiple objectsARROW-2798 - [Plasma] Use hashing function that takes into account all UniqueID bytesARROW-2802 - [Docs] Move release management guide to project wikiARROW-2804 - [Website] Link to Developer wiki (Confluence) from front pageARROW-2805 - [Python] TensorFlow import workaround not working with tensorflow-gpu if CUDA is not installedARROW-2809 - [C++] Decrease verbosity of lint checks in Travis CIARROW-2811 - [Python] Test serialization for determinismARROW-2815 - [CI] Suppress DEBUG logging when building Java library in C++ CI entriesARROW-2816 - [Python] Add __iter__ method to NativeFileARROW-2821 - [C++] Only zero memory in BooleanBuilder in one placeARROW-2822 - [C++] Zero padding bytes in PoolBuffer::ResizeARROW-2822 - [C++] Zero padding bytes in PoolBuffer::ResizeARROW-2824 - [GLib] Add garrow_decimal128_array_get_value()ARROW-2825 - [C++] Need AllocateBuffer / AllocateResizableBuffer variant with default memory poolARROW-2826 - [C++] Clarification needed between ArrayBuilder::Init(), Resize() and Reserve()ARROW-2827 - [C++] LZ4 and Zstd build may be failed in parallel buildARROW-2829 - [GLib] Add GArrowORCFileReaderARROW-2830 - [Packaging] Enable parallel build for deb package build againARROW-2832 - [Python] Pretty-print schema metadata in Schema.__repr__ARROW-2833 - [Python] Column.__repr__ will lock up Jupyter with large datasetsARROW-2834 - [GLib] Remove "enable_" prefix from Meson optionsARROW-2836 - [Packaging] Expand build matrices to multiple tasksARROW-2837 - [C++] ArrayBuilder::null_bitmap returns PoolBufferARROW-2838 - [Python] Speed up null testing with Pandas semanticsARROW-2844 - [Packaging] Test OSX wheels after buildARROW-2845 - [Packaging] Upload additional debian artifactsARROW-2846 - [Packaging] Update nightly build in crossbow as well as the sample configurationARROW-2847 - [Packaging] Fix artifact name matching for conda forge packagesARROW-2848 - [Packaging] lib*.deb package name doesn't match so versionARROW-2849 - [Ruby] Arrow::Table#load supports ORCARROW-2855 - [C++] Blog post that outlines the benefits of using jemallocARROW-2859 - [Python] Handle objects exporting the buffer protocol in open_stream, open_file, and RecordBatch*Reader APIsARROW-2861 - [Python] Add extra tips about using Parquet to store index-less pandas dataARROW-2864 - [Plasma] Add deletion cache to delete objects laterARROW-2868 - [Packaging] Fix centos-7 buildARROW-2869 - [Python] Add documentation for Array.to_numpyARROW-2874 - [Packaging] Pass job prefix when putting on QueueARROW-2875 - [Packaging] Don't attempt to download arrow archive in linux buildsARROW-2881 - [Website] Add Community tab to websiteARROW-2884 - [Packaging] Options to build packages from apache source archiveARROW-2886 - [Release] An unused variable existsARROW-2890 - [Plasma] Make Python PlasmaClient.release privateARROW-2893 - [C++] Remove PoolBuffer class from public API and hide implementation details behind factory functionsARROW-2897 - Organize supported Ubuntu versionsARROW-2898 - [Packaging] Setuptools_scm just shipped a new version which fails to parse `apache-arrow-` tag
ARROW-2906 - [Website] Remove the link to slack channelARROW-2907 - [GitHub] Improve "How to contribute patches"ARROW-2908 - [Rust] Update version to 0.10.0ARROW-2914 - [Integration] Add WindowPandasUDFTests to Spark IntegrationARROW-2915 - [Packaging] Remove artifact form ubuntu-trusty buildARROW-2918 - [C++] Improve formatting of Struct pretty printsARROW-2921 - [Release] Update .deb/.rpm changelos in preparationARROW-2922 - [Release] Make python command name customizableARROW-2923 - [Doc] Add instructions for running Spark integration testsARROW-2924 - [Java] mvn release fails when an older maven javadoc plugin is installedARROW-2927 - [Packaging] AppVeyor wheel task is failing on initial checkoutARROW-2928 - [Packaging] AppVeyor crossbow conda builds are picking up boost 1.63.0 instead of the installed versionARROW-2929 - [C++] ARROW-2826 Breaks parquet-cpp 1.4.0 buildsARROW-2934 - [Packaging] Add checksums creation to sign subcommandARROW-2935 - [Packaging] Add verify_binary_artifacts function to verify-release-candidate.shARROW-2937 - [Java] Follow-up changes to ARROW-2704ARROW-2943 - [C++] Implement BufferedOutputStream::FlushARROW-2944 - [Format] Arrow columnar format docs mentions VectorLayout that does not exist anymoreARROW-2946 - [Packaging] Stop to use PWD in debian/rulesARROW-2947 - [Packaging] Remove Ubuntu ArtfulARROW-2949 - [CI] repo.continuum.io can be flaky in buildsARROW-2951 - [CI] Changes in format/ should cause Appveyor builds to runARROW-2953 - [Plasma] Store memory usageARROW-2954 - [Plasma] Store object_id only once in object tableARROW-2962 - [Packaging] Bintray descriptor files are no longer neededARROW-2977 - [Packaging] Release verification script should check rust tooARROW-2985 - [Ruby] Run unit tests in verify-release-candidate.shARROW-2988 - [Release] More automated release verification on WindowsARROW-2990 - [GLib] Fail to build with rpath-ed Arrow C++ on macOSApache Arrow 0.9.0 (2018-03-19)
New Features and Improvements
ARROW-232 - C++/Parquet: Support writing chunked arrays as part of a tableARROW-633 - [Java] Add support for FixedSizeBinary typeARROW-634 - Add integration tests for FixedSizeBinaryARROW-760 - [Python] document differences w.r.t. fastparquetARROW-764 - [C++] Improve performance of CopyBitmap, add benchmarksARROW-969 - [C++/Python] Add add/remove field functions for RecordBatchARROW-1021 - [Python] Add documentation about using pyarrow from other Cython and C++ projectsARROW-1035 - [Python] Add ASV benchmarks for streaming columnar deserializationARROW-1394 - [Plasma] Add optional extension for allocating memory on GPUsARROW-1463 - [JAVA] Restructure ValueVector hierarchy to minimize compile-time generated codeARROW-1579 - [Java] Add dockerized test setup to validate Spark integrationARROW-1580 - [Python] Instructions for setting up nightly builds on LinuxARROW-1623 - [C++] Add convenience method to construct Buffer from a string that owns its memoryARROW-1632 - [Python] Permit categorical conversions in Table.to_pandas on a per-column basisARROW-1643 - [Python] Accept hdfs:// prefixes in parquet.read_table and attempt to connect to HDFSARROW-1705 - [Python] Create StructArray from sequence of dicts given a known data typeARROW-1706 - [Python] StructArray.from_arrays should handle sequences that are coercible to arraysARROW-1712 - [C++] Add method to BinaryBuilder to reserve space for value dataARROW-1757 - [C++] Add DictionaryArray::FromArrays alternate ctor that can check or sanitized "untrusted" indicesARROW-1815 - [Java] Rename MapVector to StructVectorARROW-1832 - [JS] Implement JSON reader for integration testsARROW-1835 - [C++] Create Arrow schema from std::tuple typesARROW-1861 - [Python] Fix up ASV setup, add developer instructions for writing new benchmarks and running benchmark suite locallyARROW-1872 - [Website] Populate hard-coded fields for current release from a YAML fileARROW-1899 - [Python] Refactor handling of null sentinels in python/numpy_to_arrow.ccARROW-1920 - Add support for reading ORC filesARROW-1926 - [GLib] Add garrow_timestamp_data_type_get_unit()ARROW-1927 - [Plasma] Implement delete functionARROW-1929 - [C++] Move various Arrow testing utility code from Parquet to Arrow codebaseARROW-1930 - [C++] Implement Slice for ChunkedArray and ColumnARROW-1931 - [C++] w4996 warning due to std::tr1 failing builds on Visual Studio 2017ARROW-1937 - [Python] Add documentation for different forms of constructing nested arrays from Python data structuresARROW-1942 - [C++] Hash table specializations for small integersARROW-1947 - [Plasma] Change Client Create and Get to use BuffersARROW-1951 - Add memcopy_threads to serialization contextARROW-1962 - [Java] Add reset() to ValueVector interfaceARROW-1965 - [GLib] Add garrow_array_builder_get_value_data_type() and garrow_array_builder_get_value_type()ARROW-1969 - [C++] Do not build ORC adapter by defaultARROW-1970 - [GLib] Add garrow_chunked_array_get_value_data_type() and garrow_chunked_array_get_value_type()ARROW-1977 - [C++] Update windows dev docsARROW-1978 - [Website] Add more visible link to "Powered By" page to front page, simplify Powered ByARROW-2004 - [C++] Add shrink_to_fit option in BufferBuilder::ResizeARROW-2007 - [Python] Sequence converter for float32 not implementedARROW-2011 - Allow setting the pickler to use in pyarrow serialization.ARROW-2012 - [GLib] Support "make distclean"ARROW-2018 - [C++] Build instruction on macOS and Homebrew is incompleteARROW-2019 - Control the memory allocated for inner vector in LISTARROW-2024 - [Python] Remove global SerializationContext variablesARROW-2028 - [Python] extra_cmake_args needs to be passed through shlex.splitARROW-2031 - HadoopFileSystem isn't pickleableARROW-2035 - [C++] Update vendored cpplint.py to a Py3-compatible oneARROW-2036 - NativeFile should support standard IOBase methodsARROW-2042 - [Plasma] Revert API change of plasma::Create to output a MutableBufferARROW-2043 - [C++] Change description from OS X to macOSARROW-2046 - [Python] Add support for PEP519 - pathlib and similar objectsARROW-2048 - [Python/C++] Upate Thrift pin to 0.11ARROW-2050 - Support `setup.py pytest` to automatically fetch the test dependenciesARROW-2052 - Unify OwnedRef and ScopedRefARROW-2053 - [C++] Build instruction is incompleteARROW-2054 - Compilation warningsARROW-2064 - [GLib] Add common build problems link to the install sectionARROW-2065 - Fix bug in SerializationContext.clone().ARROW-2066 - [Python] Document reading Parquet files from Azure Blob StoreARROW-2068 - [Python] Expose Array's buffers to Python usersARROW-2069 - [Python] Document that Plasma is not (yet) supported on WindowsARROW-2071 - [Python] Reduce runtime of builds in Travis CIARROW-2071 - [Python] Reduce runtime of builds in Travis CIARROW-2073 - [Python] Create StructArray from sequence of tuples given a known data typeARROW-2076 - [Python] Display slowest test durationsARROW-2083 - Support skipping buildsARROW-2084 - [C++] Support newer Brotli static library namesARROW-2086 - [Python] Shrink size of arrow_manylinux1_x86_64_base docker imageARROW-2087 - [Python] Binaries of 3rdparty are not stripped in manylinux1 base imageARROW-2088 - [GLib] Add GArrowNumericArrayARROW-2089 - [GLib] Rename to GARROW_TYPE_BOOLEAN for consistencyARROW-2090 - [Python] Add context manager methods to ParquetWriterARROW-2093 - [Python] Possibly do not test pytorch serialization in Travis CIARROW-2094 - [Python] Use toolchain libraries and PROTOBUF_HOME for protocol buffersARROW-2095 - [C++] Suppress ORC EP build logging by defaultARROW-2096 - [C++] Turn off Boost_DEBUG to trim build outputARROW-2099 - [Python] Support DictionaryArray::FromArrays in Python bindingsARROW-2107 - [GLib] Follow arrow::gpu::CudaIpcMemHandle API changeARROW-2108 - [Python] Update instructions for ASVARROW-2110 - [Python] Only require pytest-runner on test commandsARROW-2111 - [C++] Linting could be fasterARROW-2114 - [Python] Pull latest docker manylinux1 imageARROW-2117 - [C++] Pin clang to version 5.0ARROW-2118 - [Python] Improve error message when calling parquet.read_table on an empty fileARROW-2120 - Add possibility to use empty _MSVC_STATIC_LIB_SUFFIX for ThirdpartiesARROW-2121 - [Python] Consider special casing object arrays in pandas serializers.ARROW-2123 - [JS] Upgrade to TS 2.7.1ARROW-2132 - [Doc] Add links / mentions of Plasma store to main READMEARROW-2134 - [CI] Make Travis commit inspection more robustARROW-2137 - [Python] Don't print paths that are ignored when reading Parquet filesARROW-2138 - [C++] Have FatalLog abort instead of exitingARROW-2142 - [Python] Conversion from Numpy struct array unimplementedARROW-2143 - [Python] Provide a manylinux1 wheel for cp27mARROW-2146 - [GLib] Implement Slice for ChunkedArrayARROW-2149 - [Python] reorganize test_convert_pandas.pyARROW-2154 - [Python] __eq__ unimplemented on BufferARROW-2155 - [Python] pa.frombuffer(bytearray) returns immutable BufferARROW-2156 - [CI] Isolate Sphinx dependenciesARROW-2163 - Install apt dependencies separate from built-in Travis commands, retry on flakinessARROW-2166 - [GLib] Implement Slice for ColumnARROW-2168 - [C++] Build toolchain builds with jemallocARROW-2169 - [C++] MSVC is complaining about uncaptured variablesARROW-2174 - [JS] Export format and schema enumsARROW-2176 - [C++] Extend DictionaryBuilder to support delta dictionariesARROW-2177 - [C++] Remove support for specifying negative scale values in DecimalTypeARROW-2180 - [C++] Remove APIs deprecated in 0.8.0 releaseARROW-2181 - [Python] Add concat_tables to API reference, add documentation on useARROW-2184 - [C++] Add static constructor for FileOutputStream returning shared_ptr to base OutputStreamARROW-2185 - Remove CI directives from squashed commit messagesARROW-2190 - [GLib] Add add/remove field functions for RecordBatch.ARROW-2191 - [C++] Only use specific version of jemallocARROW-2197 - Document "undefined symbol" issue and workaroundARROW-2198 - [Python] Docstring for parquet.read_table is misleading or incorrectARROW-2199 - [JAVA] Follow up fixes for ARROW-2019. Ensure density driven capacity is never less than 1 and propagate density throughout the vector treeARROW-2203 - [C++] StderrStream classARROW-2204 - [C++] Build fails with TLS error on parquet-cpp cloneARROW-2205 - [Python] Option for integer object nullsARROW-2206 - [JS] Add Perspective as a community projectARROW-2218 - [Python] PythonFile should infer mode when not givenARROW-2231 - [CI] Use clcache on AppVeyorARROW-2238 - [C++] Detect clcache in cmake configurationARROW-2239 - [C++] Update build docs for WindowsARROW-2250 - plasma_store process should cleanup on INT and TERM signalsARROW-2252 - [Python] Create buffer from address, size and baseARROW-2253 - [Python] Support __eq__ on scalar valuesARROW-2257 - [C++] Add high-level option to toggle CXX11 ABIARROW-2261 - [GLib] Can't share the same memory in GArrowBuffer safelyARROW-2262 - [Python] Support slicing on pyarrow.ChunkedArrayARROW-2279 - [Python] Better error message if lib cannot be foundARROW-2282 - [Python] Create StringArray from buffersARROW-2283 - [C++] Support Arrow C++ installed in /usr detection by pkg-configARROW-2289 - [GLib] Add Numeric, Integer and FloatingPoint data typesARROW-2291 - [C++] README missing instructions for libboost-regex-devARROW-2292 - [Python] More consistent / intuitive name for pyarrow.frombufferARROW-2309 - [C++] Use std::make_unsignedARROW-2321 - [C++] Release verification script fails with if CMAKE_INSTALL_LIBDIR is not $ARROW_HOME/libARROW-2329 - [Website]: 0.9.0 release updateARROW-2336 - [Website] Blog post for 0.9.0 releaseARROW-2768 - [Packaging] Support Ubuntu 18.04ARROW-2783 - Importing conda-forge pyarrow fails
Bug Fixes
ARROW-1345 - [Python] Conversion from nested NumPy arrays fails on integers other than int64, float32ARROW-1589 - [C++] Fuzzing for certain input formatsARROW-1646 - [Python] pyarrow.array cannot handle NumPy scalar typesARROW-1856 - [Python] Auto-detect Parquet ABI version when using PARQUET_HOMEARROW-1909 - [C++] Bug: Build fails on windows with "-DARROW_BUILD_BENCHMARKS=ON"ARROW-1912 - [Website] Add org affiliations to committers.htmlARROW-1919 - Plasma hanging if object id is not 20 bytesARROW-1924 - [Python] Bring back pickle=True option for serializationARROW-1933 - [GLib] Build failure with --with-arrow-cpp-build-dir and GPU enabled Arrow C++ARROW-1940 - [Python] Extra metadata gets added after multiple conversions between pd.DataFrame and pa.TableARROW-1941 - Table <–> DataFrame roundtrip failingARROW-1943 - Handle setInitialCapacity() for deeply nested lists of listsARROW-1944 - FindArrow has wrong ARROW_STATIC_LIBARROW-1945 - [C++] Fix doxygen documentation of array.hARROW-1946 - Add APIs to decimal vector for writing big endian dataARROW-1948 - [Java] ListVector does not handle ipc with all non-null values with none setARROW-1950 - [Python] pandas_type in pandas metadata incorrect for List typesARROW-1953 - [JS] JavaScript builds broken on masterARROW-1955 - MSVC generates "attempting to reference a deleted function" during build.ARROW-1958 - [Python] Error in pandas conversion for datetimetz row indexARROW-1961 - [Python] Writing Parquet file with flavor='spark' loses pandas schema metadataARROW-1966 - [C++] Support JAVA_HOME paths in HDFS libjvm loading that include the jre directoryARROW-1967 - Python: AssertionError w.r.t Pandas conversion on Parquet files in 0.8.0 dev versionARROW-1971 - [Python] Add pandas serialization to the defaultARROW-1972 - Deserialization of buffer objects (and pandas dataframes) segfaults on different processes.ARROW-1973 - [Python] Memory leak when converting Arrow tables with array columns to Pandas dataframes.ARROW-1976 - [Python] Handling unicode pandas columns on parquet.read_tableARROW-1979 - [JS] JS builds handing in es2015:umd testsARROW-1980 - [Python] Race condition in `write_to_dataset`ARROW-1982 - [Python] Return parquet statistics min/max as values instead of stringsARROW-1986 - [Python] HadoopFileSystem is not picklable and cannot currently be used with multiprocessingARROW-1991 - [GLib] Docker-based documentation build is brokenARROW-1992 - [Python] to_pandas crashes when using strings_to_categoricals on empty string cols on 0.8.0ARROW-1997 - [Python] to_pandas with strings_to_categorical failsARROW-1998 - [Python] Table.from_pandas crashes when data frame is emptyARROW-1999 - [Python] from_numpy_dtype returns wrong typesARROW-2000 - Deduplicate file descriptors when plasma store replies to get request.ARROW-2002 - use pyarrow download file will raise queue.Full exceptions sometimesARROW-2003 - [Python] Do not use deprecated kwarg in pandas.core.internals.make_blockARROW-2005 - [Python] pyflakes warnings on Cython files not failing buildARROW-2008 - [Python] Type inference for int32 NumPy arrays (expecting list) returns int64 and then conversion fails
ARROW-2010 - [C++] Compiler warnings with CHECKIN warning level in ORC adapterARROW-2017 - Array initialization with large (>2**31-1) uint64 values failsARROW-2023 - [C++] Test opening IPC stream reader or file reader on an empty InputStreamARROW-2025 - [Python/C++] HDFS Client disconnect closes all open clientsARROW-2029 - [Python] Program crash on `HdfsFile.tell` if file is closedARROW-2032 - [C++] ORC ep installs on each call to ninja build (even if no work to do)ARROW-2033 - pa.array() doesn't work with iteratorsARROW-2039 - [Python] pyarrow.Buffer().to_pybytes() segfaultsARROW-2040 - [Python] Deserialized Numpy array must keep ref to underlying tensorARROW-2047 - [Python] test_serialization.py uses a python executable in PATH rather than that used for a test runARROW-2049 - ARROW-2049: [Python] Use python -m cython to run Cython, instead of CYTHON_EXECUTABLEARROW-2062 - [C++] Stalled builds in test_serialization.py in Travis CIARROW-2070 - [Python] chdir logic in setup.py buggyARROW-2072 - [Python] decimal128.byte_width crashesARROW-2080 - [Python] Update documentation after ARROW-2024ARROW-2085 - HadoopFileSystem.isdir and .isfile should return False if the path doesn't existARROW-2106 - [Python] pyarrow.array can't take a pandas Series of python datetime objects.ARROW-2109 - [C++] Boost 1.66 compilation fails on Windows on linkage stageARROW-2124 - [Python] ArrowInvalid raised if the first item of a nested list of numpy arrays is emptyARROW-2128 - [Python] Cannot serialize array of empty listsARROW-2129 - [Python] Segmentation fault on conversion of empty array to PandasARROW-2131 - [Python] Serialization test fails on Windows when library has been built in place / not installedARROW-2133 - [Python] Segmentation fault on conversion of empty nested arrays to PandasARROW-2135 - [Python] NaN values silently casted to int64 when passing explicit schema for conversion in Table.from_pandasARROW-2139 - [Python] Address Sphinx deprecation warning when building docsARROW-2145 - [Python] Decimal conversion not working for NaN valuesARROW-2150 - [Python] array equality defaults to identityARROW-2151 - [Python] Error when converting from list of uint64 arraysARROW-2153 - [C++/Python] Decimal conversion not working for exponential notationARROW-2157 - [Python] Decimal arrays cannot be constructed from Python listsARROW-2158 - [Python] Construction of Decimal array with None or np.nan failsARROW-2160 - [C++/Python] Fix decimal precision inferenceARROW-2161 - [Python] Skip test_cython_api if ARROW_HOME isn't definedARROW-2162 - [Python/C++] Decimal Values with too-high precision are multiplied by 100ARROW-2167 - [C++] Building Orc extensions fails with the default BUILD_WARNING_LEVEL=ProductionARROW-2170 - [Python] construct_metadata fails on reading files where no index was preservedARROW-2171 - [Python] OwnedRef is fragileARROW-2172 - [Python] Incorrect conversion from Numpy array when stride % itemsize != 0ARROW-2173 - [Python] NumPyBuffer destructor should hold the GILARROW-2175 - [Python] arrow_ep build is triggering during parquet-cpp build in Travis CIARROW-2178 - [JS] Fix JS html FileReader exampleARROW-2179 - [C++] arrow/util/io-util.h missing from libarrow-devARROW-2192 - Commits to master should run all builds in CI matrixARROW-2194 - [Python] Pandas columns metadata incorrect for empty string columnsARROW-2208 - [Python] install issues with jemallocARROW-2209 - [Python] Partition columns are not correctly loaded in schema of ParquetDatasetARROW-2210 - [C++] TestBuffer_ResizeOOM has a memory leak with jemallocARROW-2212 - [C++/Python] Build Protobuf in base manylinux 1 docker imageARROW-2223 - [JS] installing umd release throws an errorARROW-2227 - [Python] Table.from_pandas does not create chunked_arrays.ARROW-2228 - [Python] Unsigned int type for arrow Table not supportedARROW-2230 - [Python] JS version number is sometimes picked upARROW-2232 - [Python] pyarrow.Tensor constructor segfaultsARROW-2234 - [JS] Read timestamp low bits as Uint32sARROW-2240 - [Python] Array initialization with leading numpy nan fails with exceptionARROW-2244 - [C++] Slicing NullArray should not cause the null count on the internal data to be unknownARROW-2245 - [Python] Revert static linkage of parquet-cpp in manylinux1 wheelARROW-2246 - [Python] Use namespaced boost in manylinux1 packageARROW-2251 - [GLib] Destroying GArrowBuffer while GArrowTensor that uses the buffer causes a crashARROW-2254 - [Python] Local in-place dev versions picking up JS tagsARROW-2258 - [C++] Appveyor builds failing on masterARROW-2263 - [Python] test_cython.py fails if pyarrow is not in import path (e.g. with inplace builds)ARROW-2265 - [Python] Serializing subclasses of np.ndarray returns a np.ndarray.ARROW-2268 - Remove MD5 checksums from release processARROW-2269 - [Python] Cannot build bdist_wheel for PythonARROW-2270 - [Python] ForeignBuffer doesn't tie Python object lifetime to C++ buffer lifetimeARROW-2272 - [Python] test_plasma spams /tmpARROW-2275 - [C++] Buffer::mutable_data_ member uninitializedARROW-2280 - [Python] pyarrow.Array.buffers should also include the offsetsARROW-2284 - [Python] test_plasma error on plasma_store errorARROW-2288 - [Python] slicing logic defectiveARROW-2297 - [JS] babel-jest is not listed as a dev dependencyARROW-2304 - [C++] MultipleClients test in io-hdfs-test fails on trunkARROW-2306 - [Python] HDFS test failuresARROW-2307 - [Python] Unable to read arrow stream containing 0 record batchesARROW-2311 - [Python] Struct array slicing defectiveARROW-2312 - [JS] verify-release-candidate-sh must be updated to include JS in integration testsARROW-2313 - [GLib] Release builds must define NDEBUGARROW-2316 - [C++] Revert Buffer::mutable_data member to always inlineARROW-2318 - [C++] TestPlasmaStore.MultipleClientTest is flaky (hangs) in release buildsARROW-2320 - [C++] Vendored Boost build does not build regex libraryARROW-2406 - [Python] Segfault when creating PyArrow table from Pandas for empty string column when schema providedApache Arrow 0.8.0 (2017-12-18)
Bug Fixes
ARROW-226 - [C++] libhdfs: feedback to help determining cause of failure in opening file pathARROW-641 - [C++] Do not build/run io-hdfs-test if ARROW_HDFS=offARROW-1282 - Large memory reallocation by Arrow causes hang in jemallocARROW-1298 - C++: Add prefix to jemalloc functions to guard against issues when using multiple allocators in the same processARROW-1341 - [C++] Deprecate arrow::MakeTable in favor of new ctor from ARROW-1334ARROW-1347 - [JAVA] List null type should use consistent name for inner fieldARROW-1398 - [Python] No support reading columns of type decimal(19,4)ARROW-1409 - [Format] Use for "page" attribute in Buffer in metadataARROW-1431 - [Java] JsonFileReader doesn't intialize some vectors approperatelyARROW-1436 - PyArrow Timestamps written to Parquet as INT96 appear in Spark as 'bigint'ARROW-1540 - [C++] Fix valgrind warnings in cuda-test if possibleARROW-1541 - [C++] Race condition with arrow_gpuARROW-1543 - [C++] row_wise_conversion example doesn't correspond to ListBuilder constructor argumentsARROW-1549 - [JS] Integrate auto-generated Arrow test filesARROW-1555 - [Python] write_to_dataset on s3ARROW-1584 - [PYTHON] serialize_pandas on empty dataframeARROW-1585 - serialize_pandas round trip fails on integer columnsARROW-1586 - [PYTHON] serialize_pandas roundtrip loses columns nameARROW-1609 - Plasma: Build fails with Xcode 9.0ARROW-1615 - CXX flags for development more permissive than Travis CI buildsARROW-1617 - [Python] Do not use symlinks in python/cmake_modulesARROW-1620 - Python: Download Boost in manylinux1 build from bintrayARROW-1622 - [Plasma] Plasma doesn't compile with XCode 9ARROW-1624 - [C++] Follow up fixes / tweaks to compiler warnings for Plasma / LLVM 4.0, add to readmeARROW-1625 - [Serialization] Support OrderedDict properlyARROW-1629 - [C++] Fix problematic code paths identified by infer toolARROW-1633 - [Python] numpy "unicode" arrays not understoodARROW-1640 - Resolve OpenSSL issues in Travis CIARROW-1647 - [Plasma] Potential bug when reading/writing messages.ARROW-1653 - [Plasma] Use static cast to avoid compiler warning.ARROW-1655 - [Java] Add Scale and Precision to ValueVectorTypes.tdd for DecimalsARROW-1656 - [C++] Endianness Macro is Incorrect on Windows And MacARROW-1657 - [C++] Multithreaded Read Test Failing on Arch LinuxARROW-1658 - [Python] Out of bounds dictionary indices causes segfault after converting to pandasARROW-1663 - [Java] Follow up on ARROW-1347 and make schema backward compatibleARROW-1670 - [Python] Speed up deserialization code pathARROW-1672 - [Python] Failure to write Feather bytes columnARROW-1673 - [Python] NumPy boolean arrays get converted to uint8 arrays on NdarrayToTensor roundtripARROW-1676 - [C++] Correctly truncate oversized validity bitmaps when writing Feather formatARROW-1678 - [Python] Incorrect serialization of numpy.float16ARROW-1680 - [Python] Timestamp unit change not done in from_pandas() conversionARROW-1681 - [Python] Error writing with nulls in listsARROW-1686 - Documentation generation script creates "apidocs" directory under site/javaARROW-1693 - [JS] Error reading dictionary-encoded integration test filesARROW-1694 - [Java] Unclosed VectorSchemaRoot in JsonFileReader#readDictionaryBatches()ARROW-1695 - [Serialization] Fix reference counting of numpy arrays created in custom serialializerARROW-1698 - [JS] File reader attempts to load the same dictionary batch more than onceARROW-1704 - [GLib] Go example in test suite is brokenARROW-1708 - [JS] Linter problem breaks master buildARROW-1709 - [C++] Decimal.ToString is incorrect for negative scaleARROW-1711 - [Python] flake8 checks still not failing buildsARROW-1714 - [Python] No named pd.Series name serialized as u'None'ARROW-1720 - [Python] Segmentation fault while trying to access an out-of-bound chunkARROW-1723 - Windows: __declspec(dllexport) specified when building arrow static libraryARROW-1730 - [Python] Incorrect result from pyarrow.array when passing timestamp typeARROW-1732 - [Python] RecordBatch.from_pandas fails on DataFrame with no columns when preserve_index=FalseARROW-1735 - [C++] Cast kernels cannot write into sliced output arrayARROW-1738 - [Python] Wrong datetime conversion when pa.array with unitARROW-1739 - [Python] Fix usages of assertRaises causing broken buildARROW-1742 - C++: clang-format is not detected correct on OSX anymoreARROW-1743 - [Python] Table to_pandas fails when index contains categorical columnARROW-1745 - Compilation failure on Mac OS in plasma testsARROW-1749 - [C++] Handle range of Decimal128 values that require 39 digits to be displayedARROW-1751 - [Python] Pandas 0.21.0 introduces a breaking API change for MultiIndex constructionARROW-1754 - [Python] Fix buggy Parquet roundtrip when an index name is the same as a column nameARROW-1756 - [Python] Observed int32 overflow in Feather write/read pathARROW-1762 - [C++] unittest failure for language environmentARROW-1764 - [Python] Add -c conda-forge for Windows dev installation instructionsARROW-1766 - [GLib] Fix failing builds on OSXARROW-1768 - [Python] Fix suppressed exception in ParquetWriter.__del__ARROW-1769 - Python: pyarrow.parquet.write_to_dataset creates cyclic referencesARROW-1770 - [GLib] Fix GLib compiler warningARROW-1771 - [C++] ARROW-1749 Breaks Public API test in parquet-cppARROW-1776 - [C++[ arrow::gpu::CudaContext::bytes_allocated() isn't definedARROW-1778 - [Python] Link parquet-cpp statically, privately in manylinux1 wheelsARROW-1781 - [CI] OSX Builds on Travis-CI time out oftenARROW-1788 - Plasma store crashes when trying to abort objects for disconnected clientARROW-1791 - Integration tests generate date[DAY] values outside of reasonable rangeARROW-1793 - [Integration] fix a typo for README.mdARROW-1800 - [C++] Fix and simplify random_decimalsARROW-1805 - [Python] ignore non-parquet files when exploring datasetARROW-1811 - [C++/Python] Rename all Decimal based APIs to Decimal128ARROW-1812 - Plasma store modifies hash table while iterating during client disconnectARROW-1813 - Enforce checkstyle failure in JAVA build and fix all checkstyleARROW-1821 - Add integration test case to explicitly check for optional validity bufferARROW-1829 - [Plasma] Clean up eviction policy bookkeepingARROW-1830 - [Python] Error when loading all the files in a dictionaryARROW-1831 - [Python] Docker-based documentation build does not properly set LD_LIBRARY_PATHARROW-1836 - [C++] Fix C4996 warning from arrow/util/variant.h on MSVC buildsARROW-1839 - [C++/Python] Add Decimal Parquet Read/Write TestsARROW-1840 - [Website] The installation command failed on Windows10 anaconda environment.ARROW-1845 - [Python] Expose Decimal128TypeARROW-1852 - [Plasma] Make retrieving manager file descriptor constARROW-1853 - [Plasma] Fix off-by-one error in retry processingARROW-1863 - [Python] PyObjectStringify could render bytes-like output for more types of objectsARROW-1865 - [C++] Adding a column to an empty Table failsARROW-1869 - Fix typo in LowCostIdentityHashMapARROW-1871 - [Python/C++] Appending Python Decimals with different scales requires rescalingARROW-1873 - [Python] Segmentation fault when loading total 2GB of parquet filesARROW-1877 - Incorrect comparison in JsonStringArrayList.equalsARROW-1879 - [Python] Dask integration tests are not skipped if dask is not installedARROW-1881 - [Python] setuptools_scm picks up JS version tagsARROW-1882 - [C++] Reintroduce DictionaryBuilderARROW-1883 - [Python] BUG: Table.to_pandas metadata checking fails if columns are not presentARROW-1889 - [Python] --exclude is not available in older git versionsARROW-1890 - [Python] Masking for date32 arrays not workingARROW-1891 - [Python] NaT date32 values are only converted to nulls if from_pandas is usedARROW-1892 - [Python] Unknown list item type: binaryARROW-1893 - [Python] test_primitive_serialization fails on Python 2.7.3ARROW-1895 - [Python] Add field_name to pandas index metadataARROW-1897 - [Python] Incorrect numpy_type for pandas metadata of CategoricalsARROW-1904 - [C++] Deprecate PrimitiveArray::raw_valuesARROW-1906 - [Python] Creating a pyarrow.Array with timestamp of different unit is not castedARROW-1908 - [Python] Construction of arrow table from pandas DataFrame with duplicate column names crashesARROW-1910 - CPP README Brewfile link incorrectARROW-1914 - [C++] make -j may fail to build with -DARROW_GPU=onARROW-1915 - [Python] Parquet tests should be optionalARROW-1916 - [Java] Do not exclude java/dev/checkstyle from source releasesARROW-1917 - [GLib] Must set GI_TYPELIB_PATH in verify-release-candidate.shARROW-1935 - Download page must not link to snapshots / nightly buildsARROW-1936 - Broken links to signatures/hashes etcARROW-1939 - Correct links in release 0.8 blog post
New Features and Improvements
ARROW-480 - [Python] Add accessors for Parquet column statisticsARROW-504 - [Python] Add adapter to write pandas.DataFrame in user-selected chunk size to streaming formatARROW-507 - [C++/Python] Construct List container from offsets and values subarraysARROW-541 - [JS] Implement JavaScript-compatible implementationARROW-571 - [Python] Add APIs to build Parquet files incrementally from Arrow tablesARROW-587 - Add JIRA fix version to merge toolARROW-609 - [C++] Function for casting from days since UNIX epoch to int64 dateARROW-838 - [Python] Efficient construction of arrays from non-pandas 1D NumPy arraysARROW-905 - [Docs] Add Dockerfile for reproducible documentation generationARROW-911 - [Python] Expand development.rst with build instructions without condaARROW-942 - Support integration testing on Python 2.7ARROW-950 - [Site] Add Google Analytics tagARROW-972 - [Python] Add test cases and basic APIs for UnionArrayARROW-1032 - [JS] Support custom_metadataARROW-1047 - [Java] Add generalized stream writer and reader interfaces that are decoupled from IO / message framingARROW-1047 - [Java] Add generalized stream writer and reader interfaces that are decoupled from IO / message framingARROW-1087 - [Python] add get_include to expose directory containing header filesARROW-1114 - [C++] Create Record Batch Builder class as a reusable and efficient way to transpose row-by-row data to columnsARROW-1134 - [C++] Allow C++/CLI projects to build with ArrowARROW-1178 - [Python] Create alternative to Table.from_pandas that yields a list of RecordBatch objects with a given chunk sizeARROW-1226 - [C++] Improve / correct doxygen function documentation in arrow::ipcARROW-1250 - [Python] Define API for user type checking of array typesARROW-1362 - [Integration] Validate vector type layout in IPC messagesARROW-1367 - [Website] Divide CHANGELOG issues by component and add subheadersARROW-1369 - Support boolean types in the javascript arrow reader libraryARROW-1371 - [Website] Add "Powered By" page to the websiteARROW-1455 - [Python] Add Dockerfile for validating Dask integration outside of usual CIARROW-1471 - [JAVA] Document requirements and non/requirements for ValueVector updatesARROW-1472 - [JAVA] Design updated ValueVector Object HierarchyARROW-1473 - [JAVA] Create Prototype Code Hierarchy (Implementation Phase 1)ARROW-1474 - [JAVA] ValueVector hierarchy (Implementation Phase 2)ARROW-1476 - [JAVA] Implement final ValueVector updatesARROW-1482 - [C++] Implement casts between date32 and date64ARROW-1483 - [C++] Implement casts between time32 and time64ARROW-1484 - [C++] Implement (safe and unsafe) casts between timestamps and times of different unitsARROW-1485 - [C++] Implement union-like data type for accommodating kernel arguments which may be scalars or arraysARROW-1486 - [C++] Decide if arrow::RecordBatch needs to be copyableARROW-1487 - [C++] Implement casts from List to List, where a cast function is defined from any A to B
ARROW-1488 - [C++] Implement ArrayBuilder::Finish in terms of internal::ArrayDataARROW-1498 - [GitHub] Add CONTRIBUTING.md and ISSUE_TEMPLATE.mdARROW-1503 - [Python] Add serialization callbacks for pandas objects in pyarrow.serializeARROW-1522 - [C++] Support pyarrow.Buffer as built-in type in pyarrow.serializeARROW-1523 - [C++] Add helper data struct with methods for reading a validity bitmap possibly having a non-zero offsetARROW-1524 - [C++] More graceful solution for handling non-zero offsets on inputs and outputs in compute libraryARROW-1525 - [C++] Change functions in arrow/compare.h to not return StatusARROW-1526 - [Python] Unit tests to exercise code path in PARQUET-1100ARROW-1535 - [Python] Enable sdist source tarballs to build assuming that Arrow C++ libraries are available on the host systemARROW-1538 - [C++] Support Ubuntu 14.04 in .deb packaging automationARROW-1539 - [C++] Remove functions deprecated as of 0.7.0 and prior releasesARROW-1556 - [C++] Incorporate AssertArraysEqual function from PARQUET-1100 patchARROW-1559 - [C++] Kernel implementations for "unique" (compute distinct elements of array)ARROW-1573 - [C++] Implement stateful kernel function that uses DictionaryBuilder to compute dictionary indicesARROW-1575 - [Python] Add pyarrow.column factory functionARROW-1576 - [Python] Add utility functions (or a richer type hierachy) for checking whether data type instances are members of various type classesARROW-1577 - [JS] Package release script for NPM modulesARROW-1588 - [C++/Format] Harden Decimal FormatARROW-1593 - [PYTHON] serialize_pandas should pass through the preserve_index keywordARROW-1594 - [Python] Enable multi-threaded conversions in Table.from_pandasARROW-1600 - [C++] Zero-copy Buffer constructor from std::stringARROW-1602 - [C++] Add IsValid/IsNotNull method to arrow::ArrayARROW-1603 - [C++] Add BinaryArray method to get a value as a std::stringARROW-1604 - [Python] Support common type aliases in cast(...) and various type= argumentsARROW-1605 - [Python] pyarrow.array should be able to yield smaller integer types without an explicit castARROW-1607 - [C++] Implement DictionaryBuilder for DecimalsARROW-1613 - [Java] ArrowReader should not close the input ReadChannelARROW-1616 - [Python] Add "write" method to RecordBatchStreamWriter that dispatches to write_table/write_back as appropriateARROW-1626 - Add make targets to run the inter-procedural static analysis tool called "infer".ARROW-1627 - [JAVA] Reduce heap usage(Phase 2) - memory footprint in AllocationManager.BufferLedgerARROW-1630 - [Serialization] Support Python datetime objectsARROW-1631 - [C++] Add GRPC to ThirdpartyToolchain.cmakeARROW-1635 - Add release management guide for PMCsARROW-1637 - [C++] IPC round-trip for null typeARROW-1641 - [C++] Do not include in public headers
ARROW-1648 - C++: Add cast from Dictionary[NullType] to NullTypeARROW-1649 - C++: Print number of nulls in PrettyPrint for NullArrayARROW-1651 - [JS] Lazy row accessor in TableARROW-1652 - [JS] Separate Vector into BatchVector and CompositeVectorARROW-1654 - [Python] pa.DataType cannot be pickledARROW-1662 - Move OSX Dependency management into brew bundle BrewfilesARROW-1665 - [Serialization] Support more custom datatypes in the default serialization contextARROW-1666 - [GLib] Enable gtk-doc on Travis CI Mac environmentARROW-1667 - [GLib] Support MesonARROW-1671 - [C++] Change arrow::MakeArray to not return StatusARROW-1675 - [Python] Use RecordBatch.from_pandas in FeatherWriter.writeARROW-1677 - [Blog] Add blog post on Ray and Arrow Python serializationARROW-1679 - [GLib] Add garrow_record_batch_reader_read_next()ARROW-1683 - [Python] Restore "TimestampType" to pyarrow namespaceARROW-1684 - [Python] Simplify user API for reading nested Parquet columnsARROW-1685 - [GLib] Add GArrowTableReaderARROW-1687 - [Python] Expose UnionArray to pyarrowARROW-1689 - [Python] Categorical Indices Should Be Zero-CopyARROW-1689 - [Python] Categorical Indices Should Be Zero-CopyARROW-1690 - [GLib] Add garrow_array_is_valid()ARROW-1691 - [Java] Conform Java Decimal type implementation to format decisions in ARROW-1588ARROW-1697 - [GitHub] Add ISSUE_TEMPLATE.mdARROW-1701 - [Serialization] Support zero copy PyTorch Tensor serializationARROW-1702 - Update jemalloc in manylinux1 buildARROW-1703 - [C++] Vendor exact version of jemalloc we depend onARROW-1707 - Update dev README after movement to GitBoxARROW-1710 - [Java] Remove non-nullable vectors in new vector class hierarchyARROW-1716 - [Format/JSON] Use string integer value for Decimals in JSONARROW-1717 - [Java] Remove public static helper method in vector classes for JSONReader/WriterARROW-1718 - [Python] Implement casts from timestamp to date32/date64 and support in Array.from_pandasARROW-1719 - [Java] Remove accessor/mutatorARROW-1721 - [Python] Support null mask in places where it isn't supported in numpy_to_arrow.ccARROW-1724 - [Packaging] Support Ubuntu 17.10ARROW-1725 - [Packaging] Upload .deb for Ubuntu 17.10ARROW-1726 - [GLib] Add setup description to verify C GLib buildARROW-1727 - [Format] Expand Arrow streaming format to permit new dictionaries and deltas / additions to existing dictionariesARROW-1728 - [C++] Run clang-format checks in Travis CIARROW-1734 - C++/Python: Add cast function on Column-levelARROW-1736 - [GLib] Add GArrowCastOptions:allow-time-truncateARROW-1737 - [GLib] Use G_DECLARE_DERIVABLE_TYPEARROW-1740 - C++: Kernel to get unique values of an Array/ColumnARROW-1746 - [Python] Add build dependencies for Arch LinuxARROW-1747 - [C++] Don't export symbols of statically linked librariesARROW-1748 - [GLib] Add GArrowRecordBatchBuilderARROW-1750 - [C++] Remove the need for arrow/util/random.hARROW-1752 - [Packaging] Add GPU packages for Debian and UbuntuARROW-1753 - [Python] Provide for matching subclasses with register_type in serialization contextARROW-1755 - [C++] Add build options for MSVC to use static runtime librariesARROW-1758 - [Python] Remove pickle=True option for object serializationARROW-1759 - [Python] Add function / property to get implied Arrow schema from Parquet fileARROW-1763 - [Python] DataType should be hashableARROW-1765 - [Doc] Use dependencies from conda in C++ docker buildARROW-1767 - [C++] Support file reads and writes over 2GB on WindowsARROW-1772 - [C++] Add public-api-test module in style of parquet-cppARROW-1773 - [C++] Add casts from date/time types to compatible signed integersARROW-1775 - Ability to abort created but unsealed Plasma objectsARROW-1777 - [C++] Add static ctor ArrayData::Make for nicer syntax in placesARROW-1779 - [Java] Integration test breaks without zeroing out validity vectorsARROW-1782 - [Python] Expose compressors as pyarrow.compress, pyarrow.decompressARROW-1783 - [Python] Convert SerializedPyObject to/from sequence of component buffers with minimal memory allocation / copyingARROW-1784 - [Python] Read and write pandas.DataFrame in pyarrow.serialize by decomposing the BlockManager rather than coercing to Arrow formatARROW-1785 - [Format/C++/Java] Remove VectorLayout metadata from Flatbuffers metadataARROW-1787 - [Python] Support reading parquet files into DataFrames in a backward compatible wayARROW-1794 - [C++/Python] Rename DecimalArray to Decimal128ArrayARROW-1795 - [Plasma C++] change evict policyARROW-1801 - [Docs] Update install instructions to use red-data-tools reposARROW-1802 - [GLib] Add Arrow GPU supportARROW-1806 - [GLib] Add garrow_record_batch_writer_write_table()ARROW-1808 - [C++] Make RecordBatch interface virtual to permit record batches that lazy-materialize columnsARROW-1809 - [GLib] Use .xml instead of .sgml for GTK-Doc main fileARROW-1810 - [Plasma] Remove test shell scriptsARROW-1816 - [Java] Resolve new vector classes structure for timestamp, date and maybe intervalARROW-1817 - Configure JsonFileReader to read NaN for floatsARROW-1818 - Examine Java DependenciesARROW-1819 - [Java] Remove legacy vector classesARROW-1820 - [C++] Create arrow_compute shared library subcomponentARROW-1826 - [JAVA] Avoid branching at cell level (copyFrom)ARROW-1827 - [Java] Add checkstyle config file and header fileARROW-1828 - [C++] Implement hash kernel specialization for BooleanTypeARROW-1834 - [Doc] Build documentation in separate build foldersARROW-1838 - [C++] Use compute::Datum uniformly for input argument to kernelsARROW-1841 - [JS] Update text-encoding-utf-8 and tslib for node ESModules supportARROW-1844 - [C++] Basic benchmark suite for hash kernelsARROW-1849 - [GLib] Add input checks to GArrowRecordBatchARROW-1850 - [C++] Use const void* in Writable::Write instead of const uint8_t*ARROW-1854 - [Python] Improve performance of serializing object dtype ndarraysARROW-1855 - [GLib] Add workaround for build failure on macOSARROW-1857 - [Python] Add switch for boost linkage with static parquet in wheelsARROW-1859 - [GLib] Add GArrowDictionaryDataTypeARROW-1862 - [GLib] Add GArrowDictionaryArrayARROW-1864 - [Java] Upgrade Netty to 4.1.xARROW-1866 - [Java] Combine MapVector and NonNullableMapVector ClassesARROW-1867 - [Java] Add BitVector APIs from old vector classARROW-1874 - [GLib] Add garrow_array_unique()ARROW-1878 - [GLib] Add garrow_array_dictionary_encode()ARROW-1884 - [C++] Make JsonReader/JsonWriter classes internal APIsARROW-1885 - [Java] Restore previous MapVector class namesARROW-1901 - [Python] Support recursive mkdir for DaskFilesystemARROW-1902 - [Python] Remove mkdir race condition from write_to_datasetARROW-1905 - [Python] Add more functions for checking exact types in pyarrow.typesARROW-1911 - Add Graphistry to Arrow JS proof pointsARROW-1922 - Blog post on recent improvements/changes in JAVA VectorsARROW-1932 - [Website] Update site for 0.8.0ARROW-1934 - [Website] Blog post summarizing highlights of 0.8.0 releaseApache Arrow 0.7.1 (2017-10-01)
New Features and Improvements
ARROW-559 - Script to easily verify release in all languagesARROW-1464 - [GLib] Documentation for troubleshooting of build errorsARROW-1537 - [C++] Support building with full path install_name on macOSARROW-1546 - [GLib] Support GLib 2.40 againARROW-1548 - [GLib] Support build append in builderARROW-1578 - [C++/Python] Run lint checks in Travis CI to fail for linting issues as early as possibleARROW-1592 - [GLib] Add GArrowUIntArrayBuilderARROW-1608 - Support Release verification script on macOSARROW-1612 - [GLib] add how to install for mac os to READMEARROW-1618 - [JAVA] Reduce Heap Usage(Phase 1): move release listener logic to Allocation ManagerARROW-1634 - [Website] Updates for 0.7.1 release
Bug Fixes
ARROW-1497 - [Java] JsonFileReader doesn't set value count for some vectorsARROW-1500 - [C++] Result of ftruncate ignored in MemoryMappedFile::CreateARROW-1529 - [GLib] Fix failure on macOS on Travis CIARROW-1533 - [JAVA] realloc should consider the existing buffer capacity for computing target memory requirementARROW-1536 - [C++] Do not transitively depend on libboost_systemARROW-1542 - [C++] Windows release verification script should not modify conda environmentARROW-1544 - [JS] Export Vector type definitionsARROW-1545 - Int64Builder should not need int64() as argARROW-1547 - [JAVA] Fix 8x memory over-allocation in BitVectorARROW-1550 - [Python] Fix flaky test on WindowsARROW-1550 - [Python] Fix flaky test on WindowsARROW-1553 - [JAVA] Implement setInitialCapacity for MapWriter and pass on this capacity during lazy creation of child vectorsARROW-1554 - [Python] Document that pip wheels depend on MSVC14 runtimeARROW-1557 - [PYTHON] pyarrow.Table.from_arrays doesn't validate names lengthARROW-1590 - Flow TS Table method genericsARROW-1591 - C++: Xcode 9 is not correctly detectedARROW-1595 - [Python] Fix package dependency issues causing build failuresARROW-1598 - [C++/Tutorials] MIsmatch code comment and actual code about Object IDARROW-1601 - [C++] READ_NEXT_BITSET reads one byte past the last byte on last iterationARROW-1606 - Python: Windows wheels don't include .lib files.ARROW-1610 - C++/Python: Only call python-prefix if the default PYTHON_LIBRARY is not presentARROW-1611 - Crash in BitmapReader when length is zeroARROW-1619 - [Java] Correctly set "lastSet" for variable vectors in JsonReader
Apache Arrow 0.7.0 (2017-09-17)
Bug Fixes
ARROW-12 - Get Github activity mirrored to JIRAARROW-248 - UnionVector.close() should call clear()ARROW-269 - UnionVector getBuffers method does not include typevectorARROW-407 - BitVector.copyFromSafe() should re-allocate if necessary instead of returning falseARROW-801 - [JAVA] Provide direct access to underlying buffer memory addresses in consistent way without generating garbage or large amount indirectionsARROW-1302 - C++: ${MAKE} variable not set sometimes on older MacOS installationsARROW-1332 - [Packaging] Building Windows wheels in Apache reposARROW-1354 - [Python] Segfault in Table.from_pandas with Mixed-Type CategoriesARROW-1357 - [Python] Data corruption in reading multi-file parquet datasetARROW-1363 - [C++] IPC writer sends buffer layout for dictionary rather than indicesARROW-1365 - [Python] Remove usage of removed jemalloc_memory_pool in Python API docsARROW-1373 - [Java] Implement getBuffer() methods at the ValueVector interface
ARROW-1375 - [C++] Visual Studio 2017 Appveyor builds failingARROW-1378 - [Python] whl is not a supported wheel on this platform on Debian/JessieARROW-1379 - [Java] maven dependency issues - both unused and undeclaredARROW-1390 - [Python] Extend tests for python serializationARROW-1407 - Dictionaries can only hold a maximum of 4096 indicesARROW-1411 - [Python] Booleans in Float Columns cause SegfaultARROW-1414 - [GLib] Cast after status checkARROW-1421 - [Python] pyarrow.serialize cannot serialize a Python dict inputARROW-1426 - [Website] The title element of the top page is emptyARROW-1429 - [Python] Error loading parquet file with _metadata from HDFSARROW-1430 - [Python] flake8 warnings are not failing CI buildsARROW-1434 - [C++/Python] pyarrow.Array.from_pandas does not support datetime64[D] arraysARROW-1435 - [Python] PyArrow not propagating timezone information from Parquet to PythonARROW-1437 - [Python] pa.Array.from_pandas segfaults when given a mixed-type arrayARROW-1439 - [Packaging] Automate updating RPM in RPM buildARROW-1443 - [Java] Bug on ArrowBuf.setBytes with unsliced ByteBuffersARROW-1444 - [JAVA] BitVector.splitAndTransfer copies last byte incorrectlyARROW-1446 - Python: Writing more than 2^31 rows from pandas dataframe causes row count overflow errorARROW-1450 - [Python] Raise proper error if custom serialization handler failsARROW-1452 - [C++] Make UNUSED macro name more unique so it does not conflict with thirdparty projectsARROW-1452 - [C++] Make UNUSED macro name more unique so it does not conflict with thirdparty projectsARROW-1453 - [Python] Implement WriteTensor for non-contiguous tensorsARROW-1457 - [C++] Optimize strided WriteTensorARROW-1458 - [Python] Document that HadoopFileSystem.mkdir with create_parents=False has no effectARROW-1459 - [Python] PyArrow fails to load partitioned parquet files with non-primitive typesARROW-1461 - [C++] Disable builds using LLVM apt packages temporarilyARROW-1461 - [C++] Disable builds using LLVM apt packages temporarilyARROW-1467 - [JAVA]: Fix reset() and allocateNew() in Nullable Value Vectors templateARROW-1469 - Segfault when serialize Pandas series with mixed object typeARROW-1490 - [Java] Allow Travis CI failures for JDK9 for nowARROW-1493 - [C++] Flush the output stream at the end of each PrettyPrint functionARROW-1495 - [C++] Store shared_ptr to boxed arrays in RecordBatchARROW-1507 - [C++] arrow/compute/api.h can't be used without arrow/array.hARROW-1512 - [Docs] NumericArray has no member named 'raw_data'ARROW-1514 - [C++] Fix a typo in documentARROW-1527 - Fix Travis JDK9 buildARROW-1531 - [C++] Return ToBytes by value from Decimal128ARROW-1532 - [Python] Referencing an Empty Schema causes a SegFaultNew Features and Improvements
ARROW-34 - C++: establish a basic function evaluation modelARROW-229 - [C++] Implement safe casts for primitive typesARROW-592 - [C++] Provide .deb and .rpm packagesARROW-594 - [Python] Provide interface to write pyarrow.Table to a streamARROW-695 - Integration tests for Decimal typesARROW-696 - [C++] Add JSON read/write support for decimals for integration testsARROW-759 - [Python] Implement a transient list serialization function that can handle a mix of scalars, lists, ndarrays, dictsARROW-786 - [Format] In-memory format for 128-bit Decimals, handling of sign bitARROW-837 - [Python] Expose buffer allocation, FixedSizeBufferWriterARROW-941 - [Docs] Improve "cold start" integration testing instructionsARROW-989 - [Python] Write pyarrow.Table to FileWriter or StreamWriterARROW-1156 - [Python] pyarrow.Array.from_pandas should take a type parameterARROW-1238 - [Java] Add JSON read/write support for decimals for integration testsARROW-1286 - PYTHON: support Categorical serialization to/from parquetARROW-1307 - [Python] Add pandas serialization section + Feather API to Sphinx docsARROW-1317 - [Python] Add function to set Hadoop CLASSPATHARROW-1331 - [Java] Refactor testsARROW-1331 - [Java] Refactor testsARROW-1339 - [C++] Use boost::filesystem for handling of platform-specific file path encodingsARROW-1344 - [C++] Calling BufferOutputStream::Write after calling Finish crashesARROW-1348 - [C++/Python] Add release verification script for WindowsARROW-1351 - Automate updating CHANGELOG.md as part of release scriptsARROW-1352 - [Integration] Improve print formatting for producer, consumer lineARROW-1355 - Make arrow buildable with java9ARROW-1356 - [Website] Add new committersARROW-1358 - Update source release scripts to account for new SHA checksum policyARROW-1359 - [Python] Add Parquet writer option to normalize field names for use in SparkARROW-1364 - [C++] IPC reader and writer specialized for GPU device memoryARROW-1366 - [Python] Add instructions for starting the Plasma store when installing pyarrow from wheelsARROW-1372 - [Plasma] Support for storing data in huge pagesARROW-1376 - [C++] RecordBatchStreamReader::Open API is inconsistent with writerARROW-1377 - [Python] Add function to assist with benchmarking Parquet scan performanceARROW-1381 - [Python] Improve performance of SerializedPyObject.to_bufferARROW-1383 - [C++] Support std::vector in builder vector appends
ARROW-1384 - [C++] Add convenience function for serializing a record batch to an IPC messageARROW-1386 - [C++] Unpin CMake version in MSVC build toolchainARROW-1387 - [C++] Set up GPU leaf library build toolchainARROW-1392 - [C++] Implement reader and writer IO interfaces for GPU buffersARROW-1395 - [C++] Remove APIs deprecated as of 0.5.0 and later versionsARROW-1396 - [C++] Add PrettyPrint function for Schemas, which also outputs any dictionariesARROW-1397 - [Packaging] Use Docker instead of VagrantARROW-1399 - [C++] Add CUDA build version in a public header to help prevent ABI conflictsARROW-1400 - [Python] Ability to create partitions when writing to ParquetARROW-1401 - [C++] Add extra debugging context to failures in RETURN_NOT_OK in debug buildsARROW-1401 - [C++] Add extra debugging context to failures in RETURN_NOT_OK in debug buildsARROW-1402 - [C++] Possibly deprecate public APIs that use MutableBufferARROW-1404 - [Packaging] Build .deb and .rpm on Travis CIARROW-1405 - [Python] Add logging option for verbose memory allocationsARROW-1406 - [Python] Harden user API for generating serialized schema and record batch messages as memoryview-compatible objectsARROW-1408 - [C++] Refactor and make IPC read / write APIs more consistent, add appropriate deprecationsARROW-1410 - Plasma object store occasionally pauses for a long timeARROW-1412 - [Plasma] Add higher level API for putting and getting Python objectsARROW-1413 - [C++] Add include-what-you-use configurationARROW-1415 - [GLib] Support date32 and date64ARROW-1416 - [Format] Clarify example array in memory layout documentationARROW-1417 - [Python] Allow more generic filesystem objects to be passed to ParquetDatasetARROW-1418 - [Python] Introduce SerializationContext to register custom serialization callbacksARROW-1419 - [GLib] Suppress sign-conversion warning on ClangARROW-1427 - [GLib] Add a link to readme of Arrow GLibARROW-1428 - [C++] Append steps to clone source code to README.mbARROW-1432 - [C++] Build bundled jemalloc functions with private prefixARROW-1433 - [C++] Simplify implementation of Array::SliceARROW-1438 - [Plasma] Pull SerializationContext through PlasmaClient put and getARROW-1441 - [Site] Add Ruby to Flexible sectionARROW-1442 - [Website] Add pointer to nightly conda packages on /installARROW-1447 - [C++] Round of include-what-you-use include cleanupsARROW-1448 - [Packaging] Support uploading built .deb and .rpm to BintrayARROW-1449 - Implement Decimal using only Int128ARROW-1451 - [C++] Create arrow/io/api.hARROW-1460 - [C++] Upgrade clang-format used to LLVM 4.0ARROW-1462 - [GLib] Support time arrayARROW-1466 - [C++] Support DecimalArray in arrow::PrettyPrintARROW-1468 - [C++] Append to PrimitiveBuilder from std::vector
ARROW-1479 - [JS] Expand JavaScript implementationARROW-1480 - [Python] Improve performance of serializing setsARROW-1481 - [C++] Expose type casts as generic callable object that can write into pre-allocated memoryARROW-1494 - [C++] Document that shared_ptr returned by RecordBatch::column needs to be retainedARROW-1499 - [Python] Consider adding option to parquet.write_table that sets options for maximum Spark compatibilityARROW-1504 - [GLib] Support timestampARROW-1505 - [GLib] Simplify arguments checkARROW-1506 - [C++] Support pkg-config for compute modulesARROW-1508 - C++: Add support for FixedSizeBinaryType in DictionaryBuilderARROW-1510 - [C++] Support castARROW-1511 - [C++] Deprecate arrow::MakePrimitiveArrayARROW-1513 - C++: Add cast from Dictionary to plain arraysARROW-1515 - [GLib] Detect version directlyARROW-1516 - [GLib] Update documentARROW-1517 - Remove unnecessary temporary in DecimalUtil::ToString functionARROW-1519 - [C++] Move DecimalUtil functions to methods on the Int128 classARROW-1528 - [GLib] Resolve include dependencyARROW-1530 - [C++] Install arrow/util/parallel.hARROW-1551 - [Website] Updates for 0.7.0 releaseARROW-1597 - [Packaging] arrow-compute.pc is missing in .deb/.rpm file listApache Arrow 0.6.0 (2017-08-14)
Bug Fixes
ARROW-187 - [C++] Decide on how pedantic we want to be about exceptionsARROW-276 - [JAVA] Nullable Value Vectors should extend BaseValueVector instead of BaseDataValueVectorARROW-573 - [Python/C++] Support ordered dictionaries data, pandas CategoricalARROW-884 - [C++] Exclude internal classes from documentationARROW-932 - [Python] Fix compiler warnings on MSVCARROW-968 - [Python] RecordBatch [i:j] syntax is incompleteARROW-1192 - [JAVA] Improve splitAndTransfer performance for List and Union vectorsARROW-1195 - [C++] CpuInfo doesn't get cache size on WindowsARROW-1204 - [C++] lz4 ExternalProject fails in Visual Studio 2015ARROW-1225 - [Python] pyarrow.array does not attempt to convert bytes to UTF8 when passed a StringTypeARROW-1237 - [JAVA] Expose the ability to set lastSetARROW-1239 - issue with current version of git-commit-id-pluginARROW-1240 - security: upgrade logback to address CVE-2017-5929ARROW-1240 - security: upgrade logback to address CVE-2017-5929ARROW-1241 - [C++] Visual Studio 2017 Appveyor build jobARROW-1242 - [Java] security - upgrade Jackson to mitigate 3 CVE vulnerabilitiesARROW-1242 - [Java] security - upgrade Jackson to mitigate 3 CVE vulnerabilitiesARROW-1245 - [Integration] Java Integration Tests DisabledARROW-1248 - [Python] C linkage warnings in Clang with public Cython APIARROW-1249 - [JAVA] Expose the fillEmpties function from NullableVector.mutator
ARROW-1263 - [C++] CpuInfo should be able to get CPU features on WindowsARROW-1265 - [Plasma] Plasma store memory leak warnings in Python test suiteARROW-1267 - [Java] Handle zero length case in BitVector.splitAndTransferARROW-1269 - [Packaging] Add Windows wheel build scripts from ARROW-1068 to arrow-distARROW-1275 - [C++] Default static library prefix for Snappy should be "_static"ARROW-1276 - Cannot serializer empty DataFrame to parquetARROW-1283 - [Java] VectorSchemaRoot should be able to be closed() more than onceARROW-1285 - PYTHON: NotImplemented exception creates empty parquet fileARROW-1287 - [Python] Emulate "whence" argument of seek in NativeFileARROW-1290 - [C++] Use array capacity doubling in arrow::BufferBuilderARROW-1291 - [Python] pa.RecordBatch.from_pandas doesn't accept DataFrame with numeric column namesARROW-1294 - [C++] New Appveyor build failuresARROW-1296 - [Java] templates/FixValueVectors reset() method doesn't set allocationSizeInBytes correctlyARROW-1300 - [JAVA] Fix ListVector TestsARROW-1306 - [Python] Encoding? issue with error reporting for parquet.read_tableARROW-1308 - [C++] ld tries to link 'arrow_static' even when -DARROW_BUILD_STATIC=offARROW-1309 - [Python] Error inferring List type in Array.from_pandas when inner values are all NoneARROW-1310 - [JAVA] Revert ARROW-886ARROW-1311 - python hangs after write a few parquet tablesARROW-1312 - [C++] Set default value to ARROW_JEMALLOC to OFF until ARROW-1282 is resolvedARROW-1312 - [C++] Set default value to ARROW_JEMALLOC to OFF until ARROW-1282 is resolvedARROW-1326 - [Python] Fix Sphinx build in Travis CIARROW-1327 - [Python] Failing to release GIL in MemoryMappedFile._open causes deadlockARROW-1328 - [Python] pyarrow.Table.from_pandas option timestamps_to_ms changes column valuesARROW-1330 - [Plasma] Turn on plasma tests on manylinux1ARROW-1335 - [C++] PrimitiveArray::raw_values has inconsistent semantics re: offsets compared with subclassesARROW-1338 - [Python] Investigate non-deterministic core dump on Python 2.7, Travis CI buildsARROW-1340 - [Java] NullableMapVector field doesn't maintain metadataARROW-1342 - [Python] Support strided array of listsARROW-1343 - [Format/Java/C++] Ensuring encapsulated stream / IPC message sizes are always a multiple of 8ARROW-1350 - [C++] Include Plasma source tree in source distributionNew Features and Improvements
ARROW-439 - [Python] Add option in "to_pandas" conversions to yield Categorical from String/Binary arraysARROW-622 - [Python] Investigate alternatives to timestamps_to_ms argument in pandas conversionARROW-1076 - [Python] Handle nanosecond timestamps more gracefully when writing to Parquet formatARROW-1093 - [Python] Fail Python builds if flake8 yields warningsARROW-1104 - Integrate in-memory object store from RayARROW-1116 - [Python] Create single external GitHub repo building for building wheels for all platforms in one shotARROW-1121 - [C++] Improve error message when opening OS file failsARROW-1140 - [C++] Allow optional build of plasmaARROW-1149 - [Plasma] Create Cython client library for PlasmaARROW-1173 - [Plasma] Blog post for PlasmaARROW-1211 - [C++] Consider making default_memory_pool() the default for builder classesARROW-1213 - [Python] Enable s3fs to be used with ParquetDataset and reader/writer functionsARROW-1219 - [C++] Use more vanilla Google C++ formattingARROW-1224 - [Format] Clarify language around buffer padding and alignment in IPCARROW-1230 - [Plasma] Install libraries and headersARROW-1243 - [Java] security: upgrade all libraries to latest stable versionsARROW-1246 - [Format] Add Map logical type to metadataARROW-1251 - [Python/C++] Revise build documentation to account for latest build toolchainARROW-1253 - [C++] Use pre-built toolchain libraries where prudent to speed up CI buildsARROW-1255 - [Plasma] Check plasma flatbuffer messages with the flatbuffer verifierARROW-1256 - [Plasma] Fix compile warnings on macOSARROW-1257 - [Plasma] Plasma documentationARROW-1258 - [C++] Suppress dlmalloc warnings on ClangARROW-1259 - [Plasma] Speed up Plasma testsARROW-1260 - [Plasma] Use factory method to create Python PlasmaClientARROW-1264 - [Plasma] Don't exit the Python interpreter if the plasma client can't connect to the storeARROW-1268 - [Website] Blog post on Arrow integration with SparkARROW-1270 - [Packaging] Add Python wheel build scripts for macOS to arrow-distARROW-1272 - [Python] Add script to arrow-dist to generate and upload manylinux1 Python wheelsARROW-1273 - [Python] Add convenience functions for reading only Parquet metadata or effective Arrow schema from a particular Parquet fileARROW-1274 - [C++] add_compiler_export_flags() throws warning with CMake >= 3.3ARROW-1281 - [C++/Python] Add Docker setup for running HDFS tests and other tests we may not run in Travis CIARROW-1288 - Clean up many ASF license headersARROW-1289 - [Python] Add PYARROW_BUILD_PLASMA option like ParquetARROW-1297 - 0.6.0 ReleaseARROW-1301 - [C++/Python] Add remaining supported libhdfs UNIX-like filesystem APIsARROW-1303 - [C++] Support downloading BoostARROW-1304 - [Java] Fix checkstyle checks warningARROW-1305 - [GLib] Add GArrowIntArrayBuilderARROW-1315 - [GLib] Status check of arrow::ArrayBuilder::Finish() is missingARROW-1323 - [GLib] Add garrow_boolean_array_get_values()ARROW-1333 - [Plasma] Sorting example for DataFrames in plasmaARROW-1334 - [C++] Instantiate arrow::Table from vector of Array objects (instead of Columns)ARROW-1336 - [C++] Add arrow::schema factory functionARROW-1353 - [Website] Updates + blog post for 0.6.0 release
Apache Arrow 0.5.0 (2017-07-23)
New Features and Improvements
ARROW-111 - [C++] Add static analyzer to tool chain to verify checking of Status returnsARROW-195 - [C++] Upgrade clang bits to clang-3.8 and move back to trusty.ARROW-460 - [C++] Implement JSON round trip for DictionaryArrayARROW-462 - [C++] Implement in-memory conversions between non-nested primitive types and DictionaryArray equivalentARROW-575 - Python: Auto-detect nested lists and nested numpy arrays in PandasARROW-597 - [Python] Add convenience function to yield DataFrame from any object that a StreamReader or FileReader can read fromARROW-599 - [C++] Add LZ4 codec to 3rd-party toolchainARROW-599 - [C++] Add LZ4 codec to 3rd-party toolchainARROW-600 - [C++] Add ZSTD codec to 3rd-party toolchainARROW-692 - Java<->C++ Integration tests for dictionary-encoded vectorsARROW-693 - [Java] Add JSON support for dictionary vectorsARROW-742 - Handling exceptions during execution of std::wstring_convertARROW-742 - Handling exceptions during execution of std::wstring_convertARROW-834 - [Python] Support creating Arrow arrays from Python iterablesARROW-915 - Struct Array reads limited supportARROW-935 - [Java] Build Javadoc in Travis CIARROW-960 - [Python] Add source build guide for macOS + HomebrewARROW-962 - [Python] Add schema attribute to FileReaderARROW-964 - [Python] Improve api docsARROW-966 - [Python] pyarrow.list_ should also accept Field instanceARROW-978 - [Python] Use sphinx-bootstrap-theme for Sphinx documentationARROW-1041 - [Python] Support read_pandas on a directory of Parquet filesARROW-1048 - Allow user LD_LIBRARY_PATH to be used with source release scriptARROW-1052 - Arrow 0.5.0 releaseARROW-1071 - [Python] RecordBatchFileReader does not have a schema propertyARROW-1073 - C++: Adapative integer builderARROW-1095 - [Website] Add Arrow icon assetARROW-1100 - [Python] Add "mode" property to NativeFile instancesARROW-1102 - Make MessageSerializer.serializeMessage() publicARROW-1120 - [Python] Write support for int96ARROW-1122 - [Website] Guest blog post on Arrow + ODBC from turbodbcARROW-1122 - [Website] Guest blog post on Arrow + ODBC from turbodbcARROW-1123 - C++: Make jemalloc the default allocatorARROW-1135 - Upgrade Travis CI clang builds to use LLVM 4.0ARROW-1137 - Python: Ensure Pandas roundtrip of all-None columnARROW-1142 - [C++] Move over compression library toolchain from parquet-cppARROW-1145 - [GLib] Add get_values()ARROW-1146 - Add .gitignore for *_generated.h files in src/plasma/formatARROW-1148 - [C++] Raise minimum CMake version to 3.2ARROW-1151 - [C++] Add gcc branch prediction to status check macroARROW-1154 - [C++] Migrate more computational utility code from parquet-cppARROW-1160 - C++: Implement DictionaryBuilderARROW-1165 - [C++] Refactor PythonDecimalToArrowDecimal to not use templatesARROW-1172 - [C++] Use unique_ptr with array builder classesARROW-1183 - [Python] Implement time type conversions in to_pandasARROW-1185 - [C++] Clean up arrow::Status implementation, add warn_unused_result attribute for clangARROW-1187 - Serialize a DataFrame with None columnARROW-1193 - [C++] Support pkg-config forarrow_python.soARROW-1196 - [C++] Appveyor separate jobs for Debug/Release builds from sources; Build with conda toolchain; Build with NMake Makefiles GeneratorARROW-1198 - Python: Add public C++ API to unwrap PyArrow objectARROW-1199 - [C++] Introduce mutable POD struct for generic array dataARROW-1202 - Remove semicolons from status macrosARROW-1212 - [GLib] Add garrow_binary_array_get_offsets_buffer()ARROW-1214 - [Python] Add classes / functions to enable stream message components to be handled outside of the stream reader classARROW-1217 - [GLib] Add GInputStream based arrow::io::RandomAccessFileARROW-1220 - [C++] Standartize usage of *_HOME cmake script variables for 3rd party libsARROW-1221 - [C++] Pin clang-format versionARROW-1227 - [GLib] Support GOutputStreamARROW-1229 - [GLib] Follow Reader API change (get -> read)ARROW-1244 - [C++] Do not include cpp/src/plasma in source release pending IP clearanceARROW-1252 - [Website] Update for 0.5.0 release, add blog post summarizing changes from 0.4.x
Bug Fixes
ARROW-288 - Implement Arrow adapter for Spark DatasetsARROW-601 - Some logical types not supported when loading ParquetARROW-784 - Cleaning up thirdparty toolchain support in Arrow on WindowsARROW-785 - possible issue on writing parquet via pyarrow, subsequently read in HiveARROW-924 - Setting GTEST_HOME Fails on CMake runARROW-992 - [Python] In place development builds do not have a __version__ARROW-1043 - [Python] Make sure pandas metadata created by arrow conforms to the pandas specARROW-1074 - from_pandas doesnt convert ndarray to listARROW-1079 - [Python] Empty "private" directories should be ignored by Parquet interfaceARROW-1081 - C++: arrow::test::TestBase::MakePrimitive doesn't fill null_bitmapARROW-1096 - [C++] Memory mapping file over 4GB fails on WindowsARROW-1097 - Reading tensor needs file to be opened in writeable modeARROW-1098 - Document Error?ARROW-1101 - UnionListWriter is not implementing all methods on interface ScalarWriterARROW-1103 - [Python] Utilize pandas metadata from common _metadata Parquet file if it existsARROW-1107 - [JAVA] NullableMapVector getField() should return nullable typeARROW-1108 - Check if ArrowBuf is empty buffer in getActualConsumedMemory() and getPossibleConsumedMemory()ARROW-1109 - [JAVA] transferOwnership fails when readerIndex is not 0ARROW-1110 - [JAVA] make union vector naming consistentARROW-1111 - [JAVA] Make aligning buffers optional, and allow -1 for unknown null countARROW-1112 - [JAVA] Set lastSet for VarLength and List vectors when loadingARROW-1113 - [C++] gflags EP build gets triggered (as a no-op) on subsequent calls to make or ninja buildARROW-1115 - [C++] Use absolute path for ccacheARROW-1117 - [Docs] Minor issues in GLib READMEARROW-1124 - [Python] pyarrow needs to depend on numpy>=1.10 (not 1.9)ARROW-1125 - Python: Table.from_pandas doesn't work anymore on partial schemasARROW-1125 - Python: Table.from_pandas doesn't work anymore on partial schemasARROW-1128 - [Docs] command to build a wheel is not properly renderedARROW-1129 - [C++] Fix Linux toolchain build regression from ARROW-742ARROW-1130 - io-hdfs-test failureARROW-1131 - Python: Parquet unit tests are always skippedARROW-1132 - [Python] Unable to write pandas DataFrame w/MultiIndex containing duplicate values to parquetARROW-1136 - [C++/Python] Segfault on empty streamARROW-1138 - Travis: Use OpenJDK7 instead of OracleJDK7ARROW-1139 - [C++] dlmalloc doesn't allow arrow to be built with clang 4 or gcc 7.1.1ARROW-1141 - on import get libjemalloc.so.2: cannot allocate memory in static TLS blockARROW-1143 - C++: Fix comparison of NullArrayARROW-1144 - [C++] Remove unused variableARROW-1147 - [C++] Allow optional vendoring of flatbuffers in plasmaARROW-1150 - [C++] AdaptiveIntBuilder compiler warning on MSVCARROW-1152 - [Cython] read_tensor should work with a readable fileARROW-1153 - All non-Pandas column throws NotImplemented: unhandled typeARROW-1155 - segmentation fault when run pa.Int16Value()ARROW-1157 - C++/Python: Decimal templates are not correctly exported on OSXARROW-1159 - [C++] Static data members cannot be accessed from inline functions in Arrow headers by thirdparty usersARROW-1162 - Transfer Between Empty Lists Should Not Invoke CallbackARROW-1164 - C++: Templated functions need ARROW_EXPORT instead of ARROW_TEMPLATE_EXPORTARROW-1166 - Errors in Struct type's example and missing reference in Layout.mdARROW-1167 - [Python] Create chunked BinaryArray in Table.from_pandas when a column's data exceeds 2GBARROW-1168 - [Python] pandas metadata may contain "mixed" data typesARROW-1169 - C++: jemalloc externalproject doesn't build with CMake's ninja generatorARROW-1170 - C++: ARROW_JEMALLOC=OFF breaks linking on unittestARROW-1174 - [GLib] Investigate root cause of ListArray glib test failureARROW-1177 - [C++] Detect int32 overflow in ListBuilder::AppendARROW-1179 - C++: Add missing virtual destructorsARROW-1180 - [GLib] garrow_tensor_get_dimension_name() returns invalid addressARROW-1181 - [Python] Parquet test fail if not enabledARROW-1182 - C++: Specify BUILD_BYPRODUCTS for zlib and zstdARROW-1186 - [C++] Enable option to build arrow with minimal dependencies needed to build Parquet libraryARROW-1188 - Segfault when trying to serialize a DataFrame with Null-only Categorical ColumnARROW-1190 - VectorLoader corrupts vectors with duplicate namesARROW-1191 - [JAVA] Implement getField() method for the complex readersARROW-1194 - Getting record batch size with pa.get_record_batch_size returns a size that is too small for pandas DataFrame.ARROW-1197 - [GLib] record_batch.hpp Inclusion is missingARROW-1200 - [C++] DictionaryBuilder should use signed integers for indicesARROW-1201 - [Python] Incomplete Python types cause a core dump when repr-ingARROW-1203 - [C++] Disallow BinaryBuilder to append byte strings larger than the maximum value of int32_tARROW-1205 - C++: Reference to type objects in ArrayLoader may cause segmentation faults.ARROW-1206 - [C++] Enable MSVC builds to work with some compression library support disabledARROW-1208 - [C++] Toolchain build with ZSTD library from conda-forge failureARROW-1208 - [C++] Toolchain build with ZSTD library from conda-forge failureARROW-1215 - [Python] Class methods in API referenceARROW-1216 - Numpy arrays cannot be created from Arrow Buffers on Python 2ARROW-1218 - Arrow doesn't compile if all compression libraries are deactivatedARROW-1222 - [Python] pyarrow.array returns NullArray for array of unsupported Python objectsARROW-1223 - [GLib] Fix function name that returns wrapped objectARROW-1228 - [GLib] Test file name should be the same name as target classARROW-1233 - [C++] Validate cmake script resolving of 3rd party linked libs from correct location in toolchain buildARROW-1235 - [C++] macOS linker failure with operator<< and std::ostreamARROW-1236 - Library paths in exported pkg-config file are incorrectARROW-1284 - Windows can't install pyarrow 0.4.1 and 0.5.0
Apache Arrow 0.4.1 (2017-06-09)
Bug Fixes
ARROW-424 - [C++] Threadsafety in arrow/io/hdfs.hARROW-1039 - Python: pyarrow.Filesystem.read_parquet causing error if nthreads>1ARROW-1050 - [C++] Export arrow::ValidateArrayARROW-1051 - [Python] If pyarrow.parquet fails to import due to a shared library ABI conflict, the test_parquet.py tests silently do not runARROW-1056 - [Python] Parquet+HDFS test failure due to writing pandas indexARROW-1057 - Fix cmake warning and msvc debug assertsARROW-1060 - [Python] Add unit test for ARROW-1053ARROW-1062 - [GLib] Examples use old APIARROW-1066 - remove warning on feather for pandas >= 0.20.1ARROW-1070 - [C++] Feather files for date/time types should be written with the physical typesARROW-1075 - [GLib] Build error on macOSARROW-1082 - [GLib] Add CI on macOSARROW-1085 - [java] Follow up on template cleanup. Missing method for IntervalYearARROW-1086 - [Python] pyarrow 0.4.0 on pypi is missing pxd filesARROW-1088 - [Python] test_unicode_filename test fails when unicode filenames aren't supported by systemARROW-1090 - [Python] build_ext usabilityARROW-1091 - Decimal scale and precision are flippedARROW-1092 - More Decimal and scale flipped follow-upARROW-1094 - [C++] Incomplete buffer reads in arrow::io::ReadableFile should exactly truncate returned bufferARROW-1127 - pyarrow 4.1 import failure on Travis
New Features and Improvements
ARROW-897 - [GLib] Build arrow-glib as a separate build in the Travis CI build matrixARROW-986 - [Format] Update IPC.md to account for dictionary batchesARROW-990 - [JS] Add tslint support for linting TypeScriptARROW-1020 - [Format] Add additional language to Schema.fbs to clarify naive vs. localized Timestamp valuesARROW-1034 - [Python] Enable creation of binary wheels on Windows / MSVCARROW-1049 - [java] vector template cleanupARROW-1063 - [Website] Blog post and website updates for 0.4.0 releaseARROW-1068 - [Python] Create external repo with appveyor.yml configured for building Python wheel installersARROW-1069 - Add instructions for publishing maven artifactsARROW-1078 - [Python] Account for PARQUET-967ARROW-1080 - C++: Add tutorial about converting to/from row-wise representationARROW-1084 - Implementations of BufferAllocator should handle Netty's OutOfDirectMemoryErrorARROW-1118 - [Website] Site updates for 0.4.1
Apache Arrow 0.4.0 (2017-05-22)
Bug Fixes
ARROW-813 - [Python] setup.py sdist must also bundle dependent cmake modulesARROW-824 - Date and Time Vectors should reflect timezone-less semanticsARROW-856 - CmakeError by Unknown compiler.ARROW-909 - libjemalloc.so.2: cannot open shared object file:ARROW-939 - Fix division by zero for zero-dimensional TensorsARROW-940 - [JS] Generate multiple sets of artifactsARROW-944 - Python: Compat broken for pandas==0.18.1ARROW-948 - [GLib] Update C++ header file listARROW-952 - Compilation error on macOS with clang-802.0.42ARROW-958 - [Python] Conda build guide still needs ARROW_HOME, PARQUET_HOMEARROW-979 - [Python] Fix setuptools_scm version when release tag is not in the master timelineARROW-991 - [Python] PyArray_SimpleNew should not be used with NPY_DATETIMEARROW-995 - [Website] 0.3 release announce has a typo in referenceARROW-998 - [Doc] File format documents incorrect schema locationARROW-1003 - [C++] Hdfs and java dlls fail to load when built for Windows with MSVCARROW-1004 - ArrowInvalid: Invalid: Python object of type float is not None and is not a string, bool, or date objectARROW-1017 - Python: Table.to_pandas leaks memoryARROW-1023 - Python: Fix bundling of arrow-cpp for macOSARROW-1033 - [Python] pytest discovers scripts/test_leak.pyARROW-1045 - [JAVA] Add support for custom metadata in org.apache.arrow.vector.types.pojo.*ARROW-1046 - [Python] Conform DataFrame metadata to pandas specARROW-1053 - [Python] Memory leak with RecordBatchFileReaderARROW-1054 - [Python] Test suite fails on pandas 0.19.2ARROW-1061 - [C++] Harden decimal parsing against invalid stringsARROW-1064 - ModuleNotFoundError: No module named 'pyarrow._parquet'
New Features and Improvements
ARROW-29 - C++: Add re2 as optional 3rd-party toolchain dependencyARROW-182 - [C++] Remove Array::Validate virtual function and make a separate methodARROW-376 - Python: Convert non-range Pandas indices (optionally) to ArrowARROW-446 - [Python] Document NativeFile interfaces, HDFS client in SphinxARROW-482 - [Java] Provide API access to "custom_metadata" Field attribute in IPC settingARROW-532 - [Python] Expand pyarrow.parquet documentation for 0.3 releaseARROW-579 - Python: Provide redistributable pyarrow wheels on OSXARROW-596 - [Python] Add convenience function to convert pandas.DataFrame to pyarrow.Buffer containing a file or stream representationARROW-629 - [JS] Add unit test suiteARROW-714 - [C++] Add import_pyarrow C API in the style of NumPy for thirdparty C++ usersARROW-819 - [Python] Define public Cython APIARROW-872 - [JS] Read streaming formatARROW-873 - [JS] Implement fixed width list typeARROW-874 - [JS] Read dictionary-encoded vectorsARROW-881 - [Python] Reconstruct Pandas DataFrame indexes using custom_metadataARROW-891 - [Python] Expand Windows build instructions to not require looking at separate C++ docsARROW-899 - [Docs] Add CHANGELOG for 0.3.0ARROW-901 - [Python] Write FixedSizeBinary to ParquetARROW-913 - [Python] Only link jemalloc to the Cython extension where it's neededARROW-923 - [Docs] Generate Changelog for website with JIRA linksARROW-929 - Move KEYS file to SVN, remove from gitARROW-943 - [GLib] Support running unit tests with source archiveARROW-945 - [GLib] Add a Lua example to show Torch integrationARROW-946 - [GLib] Use "new" instead of "open" for constructor nameARROW-947 - [Python] Improve execution time of manylinux1 buildARROW-953 - Use cmake / curl from conda-forge in CI buildsARROW-954 - Make it possible to compile Arrow with header-only boostARROW-956 - remove pandas pre-0.20.0 compatARROW-957 - [Doc] Add HDFS and Windows documents to doxygen outputARROW-961 - [Python] Rename InMemoryOutputStream to BufferOutputStreamARROW-963 - [GLib] Add equalARROW-967 - [GLib] Support initializing array with bufferARROW-970 - [Python] Accidentally calling pyarrow.Table() should not segfault processARROW-977 - [java] Add Timezone aware timestamp vectorsARROW-980 - Fix detection of "msvc" COMPILER_FAMILYARROW-982 - [Website] Improve website front copy to highlight serialization efficiency benefitsARROW-984 - [GLib] Add Go examplesARROW-985 - [GLib] Update package informationARROW-988 - [JS] Add entry to Travis CI matrixARROW-993 - [GLib] Add missing error checks in Go examplesARROW-996 - [Website] Add 0.3 release announce in JapaneseARROW-997 - [Java] Implement transfer in FixedSizeListVectorARROW-1000 - [GLib] Move install document to WebsiteARROW-1001 - [GLib] Unify writer filesARROW-1002 - [C++] It is not necessary to add padding after the magic header in the FileWriter implementationARROW-1008 - [C++] Define abstract interface for stream iterationARROW-1010 - [Website] Only show English posts in /blog/ARROW-1011 - [Format] Clarify requirements around buffer padding in validity bitmapsARROW-1014 - 0.4.0 releaseARROW-1015 - [Java] Implement schema-level metadataARROW-1016 - Python: Include C++ headers (optionally) in wheelsARROW-1022 - [Python] Add nthreads option to Feather read methodARROW-1024 - Python: Update build time numpy version to 1.10.1ARROW-1025 - [Website] Improve changelog on websiteARROW-1027 - [Python] Allow negative indexing in fields/columns on pyarrow Table and Schema objectsARROW-1028 - [Python] Documentation updates after ARROW-1008ARROW-1029 - [Python] Fix --with-parquet build on Windows, add unit tests to AppveyorARROW-1030 - Python: Account for library versioning in parquet-cppARROW-1031 - [GLib] Support pretty printARROW-1037 - [GLib] Follow reader name changeARROW-1038 - [GLib] Follow writer name changeARROW-1040 - [GLib] Follow tensor IOARROW-1044 - [GLib] Support FeatherARROW-1126 - Python: Add function to convert NumPy/Pandas dtypes to Arrow DataTypes
Apache Arrow 0.3.0 (2017-05-05)
Bug Fixes
ARROW-109 - [C++] Investigate recursive data types limit in flatbuffersARROW-208 - Add checkstyle policy to java projectARROW-347 - Add method to pass CallBack when creating a transfer pairARROW-413 - DATE type is not specified clearlyARROW-431 - [Python] Review GIL release and acquisition in to_pandas conversionARROW-443 - [Python] Support for converting from strided pandas data in Table.from_pandasARROW-451 - [C++] Override DataType::Equals for other types with additional metadataARROW-454 - pojo.Field doesn't implement hashCode()ARROW-526 - [Format] Update IPC.md to account for File format changes and Streaming formatARROW-565 - [C++] Examine "Field::dictionary" memberARROW-570 - Determine Java tools JAR location from project metadataARROW-584 - [C++] Fix compiler warnings exposed with -WconversionARROW-586 - Problem with reading parquet files saved by Apache SparkARROW-588 - [C++] Fix compiler warnings on 32-bit platformsARROW-595 - [Python] StreamReader.schema returns NoneARROW-604 - Python: boxed Field instances are missing the reference to DataTypeARROW-611 - [Java] TimeVector TypeLayout is incorrectly specified as 64 bit widthARROW-613 - [JS] Implement random-access file formatARROW-617 - Time type is not specified clearlyARROW-619 - Python: Fix typos in setup.py args and LD_LIBRARY_PATHARROW-619 - Python: Fix typos in setup.py args and LD_LIBRARY_PATHARROW-623 - segfault with __repr__ of empty FieldARROW-624 - [C++] Restore MakePrimitiveArray functionARROW-627 - [C++] Compatibility macros for exported extern template class declarationsARROW-628 - [Python] Install nomkl metapackage when building parquet-cpp for faster Travis buildsARROW-630 - [C++] IPC unloading for BooleanArray does not account for offsetARROW-636 - [C++] Add Boost / other system requirements to C++ READMEARROW-639 - [C++] Invalid offset in slicesARROW-642 - [Java] Remove temporary file in java/toolsARROW-644 - Python: Cython should be a setup-only requirementARROW-652 - Remove trailing f in merge script outputARROW-654 - [C++] Support timezone metadata in file/stream formatsARROW-666 - [Python] Error in DictionaryArray __repr__ARROW-667 - build of arrow-master/cpp fails with altivec error?ARROW-668 - [Python] Convert nanosecond timestamps to pandas.Timestamp when converting from TimestampValueARROW-671 - [GLib] License file isn't installedARROW-673 - [Java] Support additional Time metadataARROW-677 - [java] Fix checkstyle jcl-over-slf4j conflict issueARROW-678 - [GLib] Fix dependenciesfffARROW-680 - [C++] Multiarch support impacts user-supplied install prefixARROW-682 - Add self-validation checks in integration testsARROW-683 - [C++] Support date32 (DateUnit::DAY) in IPC metadata, rename date to date64ARROW-685 - [GLib] AX_CXX_COMPILE_STDCXX_11 error running ./configureARROW-686 - [C++] Account for time metadata changes, add time32 and time64 typesARROW-689 - [GLib] Install header files and documents to wrong directoriesARROW-691 - [Java] Encode dictionary Int type in message formatARROW-697 - [Java] Raise appropriate exceptions when encountering large (> INT32_MAX) record batchesARROW-699 - [C++] Arrow dynamic libraries are missed on run of unit tests on WindowsARROW-702 - Fix BitVector.copyFromSafe to reAllocate instead of returning falseARROW-703 - Fix issue where setValueCount(0) doesn’t work in the case that we’ve shipped vectors across the wireARROW-704 - Fix bad import caused by conflicting changesARROW-709 - [C++] Restore type comparator for DecimalTypeARROW-713 - [C++] Fix linking issue with ipc benchmarkARROW-715 - Python: Explicit pandas import makes it a hard requirementARROW-716 - error building arrow/pythonARROW-720 - [java] arrow should not have a dependency on slf4j bridges in compileARROW-723 - Arrow freezes on write if chunk_size=0ARROW-726 - [C++] PyBuffer dtor may segfault if constructor passed an object not exporting buffer protocolARROW-732 - Schema comparison bugs in struct and union typesARROW-736 - [Python] Mixed-type object DataFrame columns should not silently coerce to an Arrow type by defaultARROW-738 - [Python] Fix manylinux1 packagingARROW-739 - Parallel build fails non-deterministically.ARROW-740 - FileReader fails for large objectsARROW-747 - [C++] Fix spurious warning caused by passing dl to add_dependenciesARROW-749 - [Python] Delete incomplete binary files when writing failsARROW-753 - [Python] Unit tests in arrow/python fail to link on some OS X platformsARROW-756 - [C++] Do not pass -fPIC when compiling with MSVCARROW-757 - [C++] MSVC build fails on googletest when using NMakeARROW-762 - Kerberos Problem with PyArrowARROW-776 - [GLib] Cast type is wrongARROW-777 - [Java] Resolve getObject behavior per changes / discussion in ARROW-729ARROW-778 - Modify merge tool to work on WindowsARROW-780 - PYTHON_EXECUTABLE Required to be set during buildARROW-781 - [Python/C++] Increase reference count for base object?ARROW-783 - Integration tests fail for length-0 record batchARROW-787 - [GLib] Fix compilation errors caused by ARROW-758ARROW-789 - Fix issue where setValueCount(0) doesn’t work in the case that we’ve shipped vectors across the wireARROW-793 - [GLib] Wrong indentARROW-794 - [C++] Check whether data is contiguous in ipc::WriteTensorARROW-796 - [Java] Checkstyle additions causing build failure in some environmentsARROW-797 - [Python] Add updated pyarrow.* public API listing in Sphinx docsARROW-800 - [C++] Boost headers being transitively included in pyarrowARROW-805 - listing empty HDFS directory returns an error instead of returning empty listARROW-809 - C++: Writing sliced record batch to IPC writes the entire arrayARROW-812 - Pip install pyarrow on mac failed.ARROW-817 - [C++] Fix incorrect code comment from ARROW-722ARROW-821 - [Python] Extra file _table_api.h generated during Python build processARROW-822 - [Python] StreamWriter fails to open with socket as sinkARROW-826 - Compilation error on Mac with -DARROW_PYTHON=onARROW-829 - Python: Parquet: Dictionary encoding is deactivated if column-wise compression was selectedARROW-830 - Python: jemalloc is not anymore publicly exposedARROW-836 - Test for timedelta compat with pandasARROW-839 - [C++] Portable alternative to PyDate_to_ms functionARROW-847 - C++: BUILD_BYPRODUCTS not specified anymore for gtestARROW-852 - Python: Also set Arrow Library PATHS when detection was done through pkg-configARROW-853 - [Python] It is no longer necessary to modify the RPATH of the Cython extensions on many environmentsARROW-858 - Remove dependency on boost regexARROW-866 - [Python] Error from file object destructorARROW-867 - [Python] Miscellaneous pyarrow MSVC fixesARROW-875 - Nullable variable length vector fillEmpties() fills an extra valueARROW-879 - compat with pandas 0.20.0ARROW-882 - [C++] On Windows statically built lib file overwrites lib file of shared buildARROW-883 - [JAVA] Introduction of new types has shifted EnumerationsARROW-885 - [Python/C++] Decimal test failure on MSVCARROW-886 - VariableLengthVectors don't reAlloc offsetsARROW-887 - [format] For backward compatibility, new unit fields must have default values matching previous implied unitARROW-888 - BitVector transfer() does not transfer ownershipARROW-895 - Nullable variable length vector lastSet not set correctlyARROW-900 - [Python] UnboundLocalError in ParquetDatasetPieceARROW-903 - [GLib] Remove a needless "."ARROW-914 - [C++/Python] Fix Decimal ToBytesARROW-922 - Allow Flatbuffers and RapidJSON to be used locally on WindowsARROW-927 - C++/Python: Add manylinux1 builds to Travis matrixARROW-928 - Update CMAKE script to detect unsupported msvc compilers versionsARROW-933 - [Python] arrow_python bindings have debug print statementARROW-934 - [GLib] Glib sources missing from result of 02-source.shARROW-936 - Fix release READMEARROW-936 - Fix release READMEARROW-938 - Fix Apache Rat errors from source release build
New Features and Improvements
ARROW-6 - Hope to add development documentARROW-39 - C++: Logical chunked arrays / columns: conforming to fixed chunk sizesARROW-52 - Set up project blogARROW-95 - Scaffold Main Documentation using asciidocARROW-98 - Java: API documentationARROW-99 - C++: Explore if RapidCheck may be helpful for testing / worth adding to toolchainARROW-183 - C++: Add storage type to DecimalTypeARROW-231 - C++: Add typed Resize to PoolBufferARROW-281 - [C++] IPC/RPC support on Win32 platformsARROW-316 - Finalize Date typeARROW-341 - [Python] Making libpyarrow available to third partiesARROW-452 - [C++/Python] Merge "Feather" file format implementationARROW-459 - [C++] Implement IPC round trip for DictionaryArray, dictionaries shared across record batchesARROW-483 - [C++/Python] Provide access to "custom_metadata" Field attribute in IPC settingARROW-491 - [C++] Add FixedWidthBinary typeARROW-492 - [C++] Add arrow/arrow.h public APIARROW-493 - [C++] Allow in-memory array over 2^31 -1 elements but require splitting at IPC / RPC boundariesARROW-502 - [C++/Python] Add MemoryPool implementation that logs allocation activity to std::coutARROW-510 - Add integration tests for date and time typesARROW-518 - C++: Make Status::OK method constexprARROW-520 - [C++] Add STL-compliant allocator that hooks into an arrow::MemoryPoolARROW-528 - [Python] Support _metadata or _common_metadata files when reading Parquet directoriesARROW-534 - [C++] Add IPC tests for date/time typesARROW-539 - [Python] Support reading Parquet datasets with standard partition directory schemesARROW-542 - [Java] Implement dictionaries in stream/file encodingARROW-550 - [Format] Add a TensorMessage typeARROW-552 - [Python] Add scalar value support for Dictionary typeARROW-557 - [Python] Explicitly opt in to HDFS unit testsARROW-563 - C++: Support non-standard gcc version stringsARROW-566 - Python: Deterministic position of libarrow in manylinux1 wheelsARROW-568 - [C++] Add default implementations for TypeVisitor, ArrayVisitor methods that return NotImplementedARROW-569 - [C++] Set version for *.pcARROW-574 - Python: Add support for nested Python lists in Pandas conversionARROW-576 - [C++] Complete round trip Union file/stream IPC testsARROW-577 - [C++] Refactor StreamWriter and FileWriter to have private implementationsARROW-578 - [C++] Add CMake option to add custom $CXXFLAGSARROW-580 - C++: Also provide jemalloc_X targets if only a static or shared version is foundARROW-582 - [Java] Add Date/Time Support to JSON FileARROW-589 - C++: Use system provided shared jemalloc if static is unavailableARROW-591 - [C++] Add round trip testing fixture for JSON formatARROW-593 - [C++] Rename ReadableFileInterface to RandomAccessFileARROW-598 - [Python] Add support for converting pyarrow.Buffer to a memoryview with zero copyARROW-603 - [C++] Add RecordBatch::Validate method that at least checks that schema matches the array metadataARROW-605 - [C++] Refactor generic ArrayLoader class, support work for Feather mergeARROW-606 - [C++] Upgrade to flatbuffers 1.6.0ARROW-608 - [Format] Days since epoch date typeARROW-610 - [C++] Win32 compatibility in file.ccARROW-612 - [Java] Field toString should show nullable flag statusARROW-615 - Move ByteArrayReadableSeekableByteChannel to vector.util packageARROW-616 - [C++] Remove -g flag in release buildsARROW-618 - [Python] Implement support for DatetimeTZ custom type from pandasARROW-620 - [C++] Add date/time support to JSON reader/writer for integration testingARROW-621 - [C++] Implement an "inline visitor" template that enables visitor-pattern-like code without virtual function dispatchARROW-625 - [C++] Add time unit to TimeType::ToStringARROW-626 - [Python] Enable pyarrow.BufferReader to read from any Python object implementing the buffer/memoryview protocolARROW-631 - [GLib] Import C API (C++ API wrapper) based on GLib from https://github.com/kou/arrow-glib ARROW-632 - [Python] Add support for FixedWidthBinary typeARROW-635 - [C++] Add JSON read/write support for FixedWidthBinaryARROW-637 - [Format] Add time zone metadata to Timestamp typeARROW-646 - Cache miniconda packagesARROW-647 - [C++] Don't require Boost static libraries to support CentOS 7ARROW-648 - [C++] Support multiarch on DebianARROW-650 - [GLib] Follow eadableFileInterface -> RnadomAccessFile changeARROW-651 - [C++] Set shared library version for .deb packagesARROW-655 - Implement DecimalArrayARROW-656 - [C++] Implement IO interface that can read and write to a fixed-size mutable bufferARROW-657 - [Python] Write and read tensors (with zero copy) into shared memoryARROW-658 - [C++] Implement in-memory arrow::Tensor objectsARROW-659 - [C++] Add multithreaded memcpy implementation (for hardware where it helps)ARROW-660 - [C++] Restore function that can read a complete encapsulated record batch messageARROW-661 - [C++] Add a Flatbuffer metadata type that supports array data over 2^31 - 1 elementsARROW-662 - [Format] Factor Flatbuffer schema metadata into a Schema.fbsARROW-663 - [Java] Support additional Time metadata + vector value accessorsARROW-664 - Make C++ Arrow serialization deterministicARROW-669 - [Python] Attach proper tzinfo when computing boxed scalars for TimestampArrayARROW-670 - Arrow 0.3 releaseARROW-672 - [Format] Bump metadata version for 0.3 releaseARROW-674 - [Java] Support additional Timestamp timezone metadataARROW-675 - [GLib] Update package metadataARROW-676 - [java] move from MinorType to FieldType in ValueVectors to carry all the relevant type bitsARROW-679 - [Format] Change RecordBatch and Field length members from int to longARROW-681 - [C++] Build Arrow on Windows with dynamically linked boostARROW-684 - Python: More informative message when parquet-cpp but not parquet-arrow is availableARROW-687 - [C++] Build and run full test suite in AppveyorARROW-688 - [C++] Use CMAKE_INSTALL_INCLUDEDIR for consistencyARROW-690 - Only send JIRA updates to issues@arrow.apache.orgARROW-698 - [C++] Add options to StreamWriter/FileWriter to permit large record batchesARROW-700 - Add headroom interface for allocator.ARROW-701 - [Java] Support additional Date metadataARROW-706 - [GLib] Add package install documentARROW-707 - Python: All none-Pandas column should be converted to NullArrayARROW-708 - [C++] Some IPC code simplification, perf analysisARROW-710 - [Python] Enable Feather APIs to read and write using Python file-like objectsARROW-711 - [C++] Remove extern template declarations for NumericArray types
ARROW-712 - [C++] Implement Array::Accept as inline visitorARROW-717 - [C++] IPC zero-copy round trips for arrow::TensorARROW-718 - [Python] Expose arrow::Tensor with conversions to/from NumPy arraysARROW-719 - [GLib] Support prepared source archive releaseARROW-722 - [Python] pandas conversions for new date and time types/metadataARROW-724 - Add "How to Contribute" section to READMEARROW-725 - [Format] Constant length list typeARROW-727 - [Python] Write memoryview-compatible objects in NativeFile.write with zero copyARROW-728 - [C++/Python] Add arrow::Table function for removing a columnARROW-729 - [Java] Add vector type for 32-bit date as days since UNIX epochARROW-731 - [C++] Add shared library related versions to .pcARROW-733 - [C++/Format] Change name of Fixed Width Binary to Fixed *Size* Binary for consistencyARROW-734 - [Python] Support for pyarrow on Windows / MSVCARROW-735 - [C++] Developer instruction document for MSVC on WindowsARROW-737 - [C++] Support obtaining mutable slices of mutable buffersARROW-741 - [Python] Add Python 3.6 to Travis CIARROW-743 - [C++] Consolidate unit tests for code in array.hARROW-744 - [GLib] Re-add an assertion to garrow_table_new() testARROW-745 - [C++] Allow use of system cpplintARROW-746 - [GLib] Add garrow_array_get_data_type()ARROW-748 - [Python] Pin runtime library versions in conda-forge packages to force upgradesARROW-751 - [Python] Rename all Cython extensions to "private" status with leading underscoreARROW-752 - [Python] Construct pyarrow.DictionaryArray from boxed pyarrow array objectsARROW-754 - [GLib] Add garrow_array_is_null()ARROW-755 - [GLib] Add garrow_array_get_value_type()ARROW-758 - [C++] Fix compiler warnings on MSVC x64ARROW-761 - [Python] Add function to compute the total size of tensor payloads, including metadata and paddingARROW-763 - C++: Use `python-config` to find libpythonX.X.dylibARROW-765 - [Python] Make generic ArrowException subclass value errorARROW-768 - [Java] Change the "boxed" object representation of date and time typesARROW-769 - [GLib] Support building without installed Arrow C++ARROW-770 - [C++] Move clang-tidy/format config files back to C++ source treeARROW-771 - [Python] Add APIs for reading individual Parquet row groupsARROW-773 - [C++] Add function to create arrow::Table with column appended to existing tableARROW-774 - [GLib] Remove needless LICENSE.txt copyARROW-775 - [Java] add simple constructors to value vectorsARROW-779 - [C++/Python] Raise exception if old metadata encounteredARROW-782 - [C++] Change struct to class for objects that meet the criteria in the Google style guideARROW-788 - Possible nondeterminism in Tensor serialization codeARROW-795 - [C++] Combine libarrow/libarrow_io/libarrow_ipcARROW-798 - [Docs] Publish Format Markdown documents somehow on arrow.apache.orgARROW-802 - [GLib] Add read examplesARROW-803 - [GLib] Update package repository URLARROW-804 - [GLib] Update build documentARROW-806 - [GLib] Support add/remove a column from tableARROW-807 - [GLib] Update "Since" tagARROW-808 - [GLib] Remove needless ignore entriesARROW-810 - [GLib] Remove io/ipc prefixARROW-811 - [GLib] Add GArrowBufferARROW-815 - [Java] Allow for expanding underlying buffer size after allocationARROW-816 - [C++] Use conda packages for RapidJSON, Flatbuffers to speed up buildsARROW-818 - [Python] Review public pyarrow.* API completeness and update docsARROW-820 - [C++] Build dependencies for Parquet library without arrow supportARROW-825 - [Python] Generalize pyarrow.from_pylist to accept any object implementing the PySequence protocolARROW-827 - [Python] Variety of Parquet improvements to support Dask integrationARROW-828 - [CPP] Document new requirement (libboost-regex-dev) in README.mdARROW-831 - Switch from boost::regex to std::regexARROW-832 - [C++] Upgrade thirdparty gtest to 1.8.0ARROW-833 - [Python] "Quickstart" build / environment setup guide for Python developersARROW-841 - [Python] Add pyarrow build to AppveyorARROW-844 - [Format] Revise format/README.md to reflect progress reaching a more complete specificationARROW-845 - [Python] Sync FindArrow.cmake changes from parquet-cppARROW-846 - [GLib] Add GArrowTensor, GArrowInt8Tensor and GArrowUInt8TensorARROW-848 - [Python] Improvements / fixes to conda quickstart guideARROW-849 - [C++] Add optional $ARROW_BUILD_TOOLCHAIN environment variable option for configuring build environmentARROW-857 - [Python] Automate publishing Python documentation to arrow-siteARROW-859 - [C++] Do not build unit tests by default?ARROW-860 - [C++] Decide if typed Tensor subclasses are worthwhileARROW-861 - [Python] Move DEVELOPMENT.md to Sphinx docsARROW-862 - [Python] Improve source build instructions in READMEARROW-863 - [GLib] Use GBytes to implement zero-copyARROW-864 - [GLib] Unify Array filesARROW-865 - [Python] Verify Parquet roundtrips for new date/time typesARROW-868 - [GLib] Use GBytes to reduce copyARROW-869 - [JS] Rename directory to js/ARROW-871 - [GLib] Unify DataType filesARROW-876 - [GLib] Unify ArrayBuffer filesARROW-877 - [GLib] Add garrow_array_get_null_bitmap()ARROW-878 - [GLib] Add garrow_binary_array_get_buffer()ARROW-880 - [GLib] Add garrow_primitive_array_get_buffer()ARROW-890 - [GLib] Add GArrowMutableBufferARROW-892 - [GLib] Fix GArrowTensor documentARROW-893 - Add GLib document to Web siteARROW-894 - [GLib] Add GArrowPoolBufferARROW-896 - [Docs] Add Jekyll plugin for including rendered Jupyter notebooks on websiteARROW-898 - [C++] Expand metadata support to field level, provide for sharing instances of KeyValueMetadataARROW-904 - [GLib] Simplify error check codesARROW-907 - C++: Convenience construct Table from schema and arraysARROW-908 - [GLib] Unify OutputStream filesARROW-910 - [C++] Write 0-length EOS indicator at end of streamARROW-916 - [GLib] Add GArrowBufferOutputStreamARROW-917 - [GLib] Add GArrowBufferReaderARROW-918 - [GLib] Use GArrowBuffer for readARROW-919 - [GLib] Use "id" to get type enum value from GArrowDataTypeARROW-920 - [GLib] Add Lua examplesARROW-925 - [GLib] Fix GArrowBufferReader testARROW-926 - Update KEYS to include wesmARROW-930 - javadoc generation fails with java 8ARROW-931 - [GLib] Reconstruct input streamARROW-965 - Website updates for 0.3.0 releaseApache Arrow 0.2.0 (2017-02-18)
Bug Fixes
ARROW-112 - [C++] Style fix for constants/enumsARROW-202 - [C++] Integrate with appveyor ci for windows support and get arrow building on windowsARROW-220 - [C++] Build conda artifacts in a build environment with better cross-linux ABI compatibilityARROW-224 - [C++] Address static linking of boost dependenciesARROW-230 - Python: Do not name modules like native ones (i.e. rename pyarrow.io)ARROW-239 - [Python] HdfsFile.read called with no arguments should read remainder of fileARROW-261 - [C++] Refactor BinaryArray/StringArray classes to not inherit from ListArrayARROW-273 - Lists use unsigned offset vectors instead of signed (as defined in the spec)ARROW-275 - Add tests for UnionVector in Arrow FileARROW-294 - [C++] Do not use fopen / fclose / etc. methods for memory mapped file implementationARROW-322 - [C++] Do not build HDFS IO interface optionallyARROW-323 - [Python] Opt-in to PyArrow parquet build rather than skipping silently on failureARROW-334 - [Python] OS X rpath issues on some configurationsARROW-337 - UnionListWriter.list() is doing more than it should, this can cause data corruptionARROW-339 - Make merge_arrow_pr script work with Python 3ARROW-339 - Make merge_arrow_pr script work with Python 3ARROW-340 - [C++] Opening a writeable file on disk that already exists does not truncate to zeroARROW-342 - Set Python version on releaseARROW-345 - libhdfs integration doesn't work for MacARROW-346 - Python API DocumentationARROW-348 - [Python] CMake build type should be configurable on the command lineARROW-349 - Six is missing as a requirement in the python setup.pyARROW-351 - Time type has no unitARROW-354 - Connot compare an array of empty strings to anotherARROW-357 - Default Parquet chunk_size of 64k is too smallARROW-358 - [C++] libhdfs can be in non-standard locations in some Hadoop distributionsARROW-362 - Python: Calling to_pandas on a table read from Parquet leaks memoryARROW-371 - Python: Table with null timestamp becomes float in pandasARROW-375 - columns parameter in parquet.read_table() raises KeyError for valid columnARROW-384 - Align Java and C++ RecordBatch data and metadata layoutARROW-386 - [Java] Respect case of struct / map field namesARROW-387 - [C++] arrow::io::BufferReader does not permit shared memory ownership in zero-copy readsARROW-390 - C++: CMake fails on json-integration-test with ARROW_BUILD_TESTS=OFFARROW-392 - Fix string/binary integration testsARROW-393 - [JAVA] JSON file reader fails to set the buffer size on String data vectorARROW-395 - Arrow file format writes record batches in reverse order.ARROW-398 - [Java] Java file format requires bitmaps of all 1's to be written when there are no nullsARROW-399 - [Java] ListVector.loadFieldBuffers ignores the ArrowFieldNode length metadataARROW-400 - [Java] ArrowWriter writes length 0 for Struct typesARROW-401 - [Java] Floating point vectors should do an approximate comparison in integration testsARROW-402 - [Java] "refCnt gone negative" error in integration testsARROW-403 - [JAVA] UnionVector: Creating a transfer pair doesn't transfer the schema to destination vectorARROW-404 - [Python] Closing an HdfsClient while there are still open file handles results in a crashARROW-405 - [C++] Be less stringent about finding include/hdfs.h in HADOOP_HOMEARROW-406 - [C++] Large HDFS reads must utilize the set file buffer size when making RPCsARROW-408 - [C++/Python] Remove defunct conda recipesARROW-414 - [Java] "Buffer too large to resize to ..." errorARROW-420 - Align Date implementation between Java and C++ARROW-421 - [Python] Zero-copy buffers read by pyarrow::PyBytesReader must retain a reference to the parent PyBytes to avoid premature garbage collection issuesARROW-422 - C++: IPC should depend on rapidjson_ep if RapidJSON is vendoredARROW-429 - git-archive SHA-256 checksums are changingARROW-433 - [Python] Date conversion is locale-dependentARROW-434 - Segfaults and encoding issues in Python Parquet readsARROW-435 - C++: Spelling mistake in if(RAPIDJSON_VENDORED)ARROW-437 - [C++] clang compiler warnings from overridden virtual functionsARROW-445 - C++: arrow_ipc is built before arrow/ipc/Message_generated.h was generatedARROW-447 - Python: Align scalar/pylist string encoding with pandas' one.ARROW-455 - [C++] BufferOutputStream dtor does not call Close()ARROW-469 - C++: Add option so that resize doesn't decrease the capacityARROW-481 - [Python] Fix Python 2.7 regression in patch for PARQUET-472ARROW-486 - [C++] arrow::io::MemoryMappedFile can't be casted to arrow::io::FileInterfaceARROW-487 - Python: ConvertTableToPandas segfaults if ObjectBlock::Write failsARROW-494 - [C++] When MemoryMappedFile is destructed, memory is unmapped even if buffer referecnes still existARROW-499 - Update file serialization to use streaming serialization formatARROW-505 - [C++] Fix compiler warnings in release modeARROW-511 - [Python] List[T] conversions not implemented for single arraysARROW-513 - [C++] Fix Appveyor buildARROW-516 - Building pyarrow with parquetARROW-519 - [C++] Missing vtable in libarrow.dylib on Xcode 6.4ARROW-523 - Python: Account for changes in PARQUET-834ARROW-533 - [C++] arrow::TimestampArray / TimeArray has a broken constructorARROW-535 - [Python] Add type mapping for NPY_LONGLONGARROW-537 - [C++] StringArray/BinaryArray comparisons may be incorrect when values with non-zero length are nullARROW-540 - [C++] Fix build in aftermath of ARROW-33ARROW-543 - C++: Lazily computed null_counts counts number of non-null entriesARROW-544 - [C++] ArrayLoader::LoadBinary fails for length-0 arraysARROW-545 - [Python] Ignore files without .parq or .parquet prefix when reading directory of filesARROW-548 - [Python] Add nthreads option to pyarrow.Filesystem.read_parquetARROW-551 - C++: Construction of Column with nullptr Array segfaultsARROW-556 - [Integration] Can not run Integration tests if different cpp build pathARROW-561 - Update java & python dependencies to improve downstream packaging experienceARROW-562 - Mockito should be in test scope
New Features and Improvements
ARROW-33 - C++: Implement zero-copy array slicingARROW-81 - [Format] Add a Category logical type (distinct from dictionary-encoding)ARROW-96 - C++: API documentation using DoxygenARROW-97 - Python: API documentation via sphinx-apidocARROW-108 - [C++] Add IPC round trip for union typesARROW-189 - C++: Use ExternalProject to build thirdparty dependenciesARROW-191 - Python: Provide infrastructure for manylinux1 wheelsARROW-221 - Add switch for writing Parquet 1.0 compatible logical typesARROW-227 - [C++/Python] Hook arrow_io generic reader / writer interface into arrow_parquetARROW-228 - [Python] Create an Arrow-cpp-compatible interface for reading bytes from Python file-like objectsARROW-240 - Installation instructions for pyarrowARROW-243 - [C++] Add "driver" option to HdfsClient to choose between libhdfs and libhdfs3 at runtimeARROW-268 - [C++] Flesh out union implementation to have all required methods for IPCARROW-303 - [C++] Also build static libraries for leaf librariesARROW-312 - [Python] Provide Python API to read/write the Arrow IPC file formatARROW-312 - [Python] Provide Python API to read/write the Arrow IPC file formatARROW-317 - [C++] Implement zero-copy Slice method on arrow::Buffer that retains reference to parentARROW-327 - [Python] Remove conda builds from Travis CI processesARROW-328 - [C++] Return shared_ptr by value instead of const-ref?ARROW-330 - [C++] CMake functions to simplify shared / static library configurationARROW-332 - [Python] Add helper function to convert RecordBatch to pandas.DataFrameARROW-333 - Make writers update their internal schema even when no data is written.ARROW-335 - Improve Type apis and toString() by encapsulating flatbuffers betterARROW-336 - Run Apache Rat in Travis buildsARROW-338 - [C++] Refactor IPC vector "loading" and "unloading" to be based on cleaner visitor patternARROW-344 - Instructions for building with condaARROW-350 - Add Kerberos support to HDFS shimARROW-353 - Arrow release 0.2ARROW-355 - Add tests for serialising arrays of empty strings to ParquetARROW-356 - Add documentation about reading ParquetARROW-359 - Need to document ARROW_LIBHDFS_DIRARROW-360 - C++: Add method to shrink PoolBuffer using reallocARROW-361 - Python: Support reading a column-selection from Parquet filesARROW-363 - Set up Java/C++ integration test harnessARROW-365 - Python: Provide Array.to_pandas()ARROW-366 - [java] implement Dictionary vectorARROW-367 - [java] converter csv/json <=> Arrow file format for Integration testsARROW-368 - Document use of LD_LIBRARY_PATH when using PythonARROW-369 - [Python] Add ability to convert multiple record batches at once to pandasARROW-370 - Python: Pandas conversion from `datetime.date` columnsARROW-372 - Create JSON arrow file format for integration testsARROW-373 - [C++] Implement C++ version of JSON file format for testingARROW-374 - Python: clarify unicode vs. binary in APIARROW-377 - Python: Add support for conversion of Pandas.CategoricalARROW-379 - Python: Use setuptools_scm/setuptools_scm_git_archive to provide the version numberARROW-380 - [Java] optimize null count when serializing vectors.ARROW-381 - [C++] Simplify primitive array type builders to use a default type singletonARROW-382 - Python: Extend API documentationARROW-383 - [C++] Implement C++ version of ARROW-367 integration test validatorARROW-389 - Python: Write Parquet files to pyarrow.io.NativeFile objectsARROW-394 - Add integration tests for boolean, list, struct, and other basic typesARROW-396 - Python: Add pyarrow.schema.Schema.equalsARROW-409 - Python: Change pyarrow.Table.dataframe_from_batches API to create Table insteadARROW-410 - [C++] Add Flush method to arrow::io::OutputStreamARROW-411 - [Java] Move Intergration.compare and Intergration.compareSchemas to a public utils classARROW-415 - C++: Add Equals implementation to compare TablesARROW-416 - C++: Add Equals implementation to compare ColumnsARROW-417 - C++: Add Equals implementation to compare ChunkedArraysARROW-418 - [C++] Consolidate array container and builder code, remove arrow/typesARROW-419 - [C++] Promote util/{status.h, buffer.h, memory-pool.h} to top level of arrow/ source directoryARROW-423 - C++: Define BUILD_BYPRODUCTS in external project to support non-make CMake generatorsARROW-425 - Python: Expose a C function to convert arrow::Table to pyarrow.TableARROW-426 - Python: Conversion from pyarrow.Array to a Python listARROW-427 - [C++] Implement dictionary-encoded array containerARROW-428 - [Python] Deserialize from Arrow record batches to pandas in parallel using a thread poolARROW-430 - Python: Better version handlingARROW-432 - [Python] Avoid unnecessary memory copy in to_pandas conversion by using low-level pandas internals APIsARROW-438 - [Python] Concatenate Table instances with equal schemasARROW-440 - [C++] Support pkg-configARROW-441 - [Python] Expose Arrow's file and memory map classes as NativeFile subclassesARROW-442 - [Python] Add public Python API to inspect Parquet file metadataARROW-444 - [Python] Avoid unnecessary memory copies from use of PyBytes_* C APIsARROW-449 - Python: Conversion from pyarrow.{Table,RecordBatch} to a Python dictARROW-450 - Python: Fixes for PARQUET-818ARROW-456 - C++: Add jemalloc based MemoryPoolARROW-457 - Python: Better control over memory poolARROW-458 - [Python] Expose jemalloc MemoryPoolARROW-461 - [Python] Implement conversion between arrow::DictionaryArray and pandas.CategoricalARROW-463 - C++: Support jemalloc 4.xARROW-466 - C++: ExternalProject for jemallocARROW-467 - [Python] Run parquet-cpp unit tests in Travis CIARROW-468 - Python: Conversion of nested data in pd.DataFrames to/from Arrow structuresARROW-470 - [Python] Add "FileSystem" abstraction to access directories of files in a uniform wayARROW-471 - [Python] Enable ParquetFile to pass down separately-obtained file metadataARROW-472 - [Python] Expose parquet::{SchemaDescriptor, ColumnDescriptor}::EqualsARROW-474 - Create an Arrow streaming file fomatARROW-475 - [Python] High level support for reading directories of Parquet files (as a single Arrow table) from supported file system interfacesARROW-476 - [Integration] Add integration tests for Binary / Varbytes typeARROW-477 - [Java] Add support for second/microsecond/nanosecond timestamps in-memory and in IPC/JSON layerARROW-478 - [Python] Accept a PyBytes object in the pyarrow.io.BufferReader ctorARROW-479 - Python: Test for expected schema in Pandas conversionARROW-484 - Add more detail about what of technology can be found in the Arrow implementations to READMEARROW-485 - [Java] Users are required to initialize VariableLengthVectors.offsetVector before calling VariableLengthVectors.mutator.getSafeARROW-490 - Python: Update manylinux1 build scriptsARROW-495 - [C++] Add C++ implementation of streaming serialized formatARROW-497 - [Java] Integration test harness for streaming formatARROW-498 - [C++] Integration test harness for streaming formatARROW-503 - [Python] Interface to streaming binary formatARROW-506 - Implement Arrow Echo server for integration testingARROW-508 - [C++] Make file/memory-mapped file interfaces threadsafeARROW-509 - [Python] Add support for PARQUET-835 (parallel column reads)ARROW-512 - C++: Add method to check for primitive typesARROW-514 - [Python] Accept pyarrow.io.Buffer as input to StreamReader, FileReader classesARROW-515 - [Python] Add StreamReader/FileReader methods that read all record batches as a TableARROW-521 - [C++/Python] Track peak memory use in default MemoryPoolARROW-524 - [java] provide apis to access nested vectors and buffersARROW-525 - Python: Add more documentation to the packageARROW-527 - clean drill-module.conf fileARROW-529 - Python: Add jemalloc and Python 3.6 to manylinux1 buildARROW-531 - Python: Document jemalloc, extend Pandas section, add Getting InvolvedARROW-538 - [C++] Set up AddressSanitizer (ASAN) buildsARROW-546 - Python: Account for changes in PARQUET-867ARROW-547 - [Python] Expose Array::Slice and RecordBatch::SliceARROW-553 - C++: Faster valid bitmap buildingARROW-558 - Add KEYS files
Apache Arrow 0.1.0 (2016-10-10)
New Features and Improvements
ARROW-1 - Import Initial CodebaseARROW-2 - Post Simple WebsiteARROW-3 - Post Initial Arrow Format SpecARROW-4 - Initial Arrow CPP ImplementationARROW-7 - Add Python library build toolchainARROW-8 - Set up Travis CIARROW-9 - Rename some unchanged "Drill" to "Arrow"ARROW-9 - Rename some unchanged "Drill" to "Arrow"ARROW-10 - Fix mismatch of javadoc names and method parametersARROW-11 - Mirror JIRA activity to dev@arrow.apache.orgARROW-13 - Add PR merge tool similar to that used in ParquetARROW-14 - Add JIRA componentsARROW-15 - Fix a naming typo for memory.AllocationManager.AllocationOutcomeARROW-19 - C++: Externalize memory allocations and add a MemoryPool abstract interface to builder classesARROW-20 - C++: Add null count member to Array containers, remove nullable memberARROW-21 - C++: Add in-memory schema metadata containerARROW-22 - C++: Add schema adapter routines for converting flat Parquet schemas to in-memory Arrow schemasARROW-23 - C++: Add logical "Column" container for chunked dataARROW-24 - C++: Add logical "Table" containerARROW-26 - C++: Add developer instructions for building parquet-cpp integrationARROW-28 - C++: Add google/benchmark to the 3rd-party build toolchainARROW-30 - Python: pandas/NumPy to/from Arrow conversion routinesARROW-31 - Python: basic PyList <-> Arrow marshaling codeARROW-35 - Add a short call-to-action / how-to-get-involved to the main README.mdARROW-37 - C++: Represent boolean array data in bit-packed formARROW-42 - Python: Add to Travis CI buildARROW-43 - Python: Add rudimentary console __repr__ for array typesARROW-44 - Python: Implement basic object model for scalar values (i.e. results of arrow_arr[i])ARROW-48 - Python: Add Schema object wrapperARROW-49 - Python: Add Column and Table wrapper interfaceARROW-50 - C++: Enable library builds for 3rd-party users without having to build thirdparty googletestARROW-53 - Python: Fix RPATH and add source installation instructionsARROW-54 - Python: rename package to "pyarrow"ARROW-56 - Format: Specify LSB bit ordering in bit arraysARROW-57 - Format: Draft data headers IDL for data interchangeARROW-58 - Format: Draft type metadata ("schemas") IDLARROW-59 - Python: Boolean data support for builtin data structuresARROW-60 - C++: Struct type builder APIARROW-64 - Add zsh support to C++ build scriptsARROW-66 - Maybe some missing steps in installation guideARROW-67 - C++: Draft type metadata conversion to/from IPC representationARROW-68 - Update setup_build_env and third-party script to be more userfriendlyARROW-70 - C++: Add "lite" DCHECK macros used in parquet-cppARROW-71 - C++: Add script to run clang-tidy on codebaseARROW-73 - Support CMake 2.8ARROW-76 - Revise format document to include null count, defer non-nullable arrays to the domain of metadataARROW-78 - C++: Add constructor for DecimalTypeARROW-79 - Python: Add benchmarksARROW-82 - C++: Implement IPC exchange for List typesARROW-85 - C++: memcmp can be avoided in Equal when comparing with the same BufferARROW-86 - Python: Implement zero-copy Arrow-to-Pandas conversionARROW-87 - Implement Decimal schema conversion for all ways supported in ParquetARROW-89 - Python: Add benchmarks for Arrow<->Pandas conversionARROW-90 - Apache Arrow cpp code does not support power architectureARROW-91 - C++: First draft of an adapter class for parquet-cpp's ParquetFileReader that produces Arrow table/row batch objectsARROW-92 - C++: Arrow to Parquet Schema conversionARROW-100 - [C++] Computing RowBatch sizeARROW-101 - Fix java warnings emitted by java compilerARROW-102 - travis-ci support for java projectARROW-106 - Add IPC round trip for string types (string, char, varchar, binary)ARROW-107 - [C++] add ipc round trip for struct typesARROW-190 - Python: Provide installable sdist buildsARROW-196 - [C++] Add conda dev recipe for libarrow and libarrow_parquetARROW-197 - [Python] Add conda dev recipe for pyarrowARROW-199 - [C++] Refine third party dependencyARROW-201 - C++: Initial ParquetWriter implementationARROW-203 - Python: Basic filename based Parquet read/writeARROW-204 - [Python] Automate uploading conda build artifacts for libarrow and pyarrowARROW-206 - [C++] Expose an equality API for arrays that compares a range of slots on two arraysARROW-207 - Extend BufferAllocator interface to allow decorators around BufferAllocatorARROW-212 - [C++] Clarify the fact that PrimitiveArray is now abstract classARROW-213 - Exposing static arrow buildARROW-214 - C++: Add String support to Parquet I/OARROW-215 - C++: Support other integer types in Parquet I/OARROW-218 - Add option to use GitHub API token via environment variable when merging PRsARROW-222 - [C++] Create prototype file-like interface to HDFS (via libhdfs) and begin defining more general IO interface for Arrow data adaptersARROW-233 - [C++] Add visibility defines for limiting shared library symbol visibilityARROW-234 - [C++] Build with libhdfs support in arrow_io in conda buildsARROW-236 - [Python] Enable Parquet read/write to work with HDFS file objectsARROW-237 - [C++] Create Arrow specializations of Parquet allocator and read interfacesARROW-238 - C++: InternalMemoryPool::Free() should throw an error when there is insufficient allocated memoryARROW-242 - C++/Python: Support Timestamp Data TypeARROW-245 - [Format] Clarify Arrow's relationship with big endian platformsARROW-251 - [C++] Expose APIs for getting code and message of the statusARROW-252 - Add implementation guidelines to the documentationARROW-253 - Int types should only have width of 8*2^n (8, 16, 32, 64)ARROW-254 - Remove Bit type as it is redundant with booleanARROW-255 - Finalize Dictionary representationARROW-256 - Add versioning to the arrow spec.ARROW-257 - Add a typeids Vector to Union typeARROW-262 - [Format] Add a new format document for metadata and logical types for messaging and IPC / on-wire/file representationsARROW-264 - Create an Arrow File formatARROW-267 - [C++] C++ implementation of file-like layout for RPC / IPCARROW-270 - [Format] Define more generic Interval logical typeARROW-271 - Update Field structure to be more explicitARROW-272 - Arrow release 0.1ARROW-279 - rename vector module to arrow-vector for consistencyARROW-280 - [C++] Consolidate file and shared memory IO interfacesARROW-282 - Make parquet-cpp an optional dependency of pyarrowARROW-285 - Allow for custom flatc compilerARROW-286 - Build thirdparty dependencies in parallelARROW-289 - Install test-util.hARROW-290 - Specialize alloc() in ArrowBufARROW-291 - [Python] Update NOTICE file for Python codebaseARROW-292 - [Java] Upgrade Netty to 4.041ARROW-293 - [C++] Implementations of IO interfaces for operating system filesARROW-296 - [C++] Remove arrow_parquet C++ module and related parts of build systemARROW-298 - create release scriptsARROW-299 - Use absolute namespace in macrosARROW-301 - [Format] Add some form of user field metadata to IPC schemasARROW-302 - [Python] Add support to use the Arrow file format with file-like objectsARROW-305 - Add compression and use_dictionary options to Parquet interfaceARROW-306 - Add option to pass cmake arguments via environment variableARROW-315 - Finalize timestamp typeARROW-318 - [Python] Revise README to reflect current state of projectARROW-319 - Add canonical Arrow Schema json representationARROW-324 - Update arrow metadata diagramARROW-325 - make TestArrowFile not dependent on timezone
Bug Fixes
ARROW-5 - Error when run maven installARROW-5 - Error when run maven installARROW-16 - Building cpp issues on XCode 7.2.1ARROW-17 - Set some vector fields to default access level for Drill compatibilityARROW-18 - Fix bug with decimal precision and scaleARROW-36 - Remove fixVersions from patch tool (until we have them)ARROW-46 - Port DRILL-4410 to ArrowARROW-51 - Move ValueVector test from Drill projectARROW-55 - Python: fix legacy Python (2.7) tests and add to Travis CIARROW-62 - Format: Are the nulls bits 0 or 1 for null values?ARROW-63 - C++: ctest fails if Python 3 is the active Python interpreterARROW-65 - Python: FindPythonLibsNew does not work in a virtualenvARROW-69 - Change permissions for assignable usersARROW-72 - FindParquet searches for non-existent headerARROW-75 - C++: Fix handling of empty stringsARROW-77 - C++: conform null bit interpretation to match ARROW-62ARROW-80 - Segmentation fault on len(Array) for empty arraysARROW-83 - Add basic test infrastructure for DecimalTypeARROW-84 - C++: separate test codesARROW-88 - C++: Refactor given PARQUET-572ARROW-93 - XCode 7.3 breaks buildsARROW-94 - Expand list example to clarify null vs empty listARROW-103 - Missing patterns from .gitignoreARROW-104 - Update Layout.md based on discussion on the mailing listARROW-105 - Unit tests fail if assertions are disabledARROW-113 - TestValueVector test fails if cannot allocate 2GB of memoryARROW-185 - [C++] Make sure alignment and memory padding conform to specARROW-188 - Python: Add numpy as install requirementARROW-193 - For the instruction, typos "int his" should be "in this"ARROW-194 - C++: Allow read-only memory mapped sourceARROW-200 - [Python] Convert Values String looks like it has incorrect error handlingARROW-205 - builds failing on master branch with apt-get errorARROW-209 - [C++] Broken builds: llvm.org apt repos are unavailableARROW-210 - [C++] Tidy up the type system a little bitARROW-211 - Several typos/errors in Layout.md examplesARROW-217 - Fix Travis w.r.t conda 4.1.0 changesARROW-219 - [C++] Passed CMAKE_CXX_FLAGS are being dropped, fix compiler warningsARROW-223 - Do not link against libpythonARROW-225 - [C++/Python] master Travis CI build is brokenARROW-244 - [C++] Some global APIs of IPC module should be visible to the outsideARROW-246 - [Java] UnionVector doesn't call allocateNew() when creating it's vectorTypeARROW-247 - [C++] Missing explicit destructor in RowBatchReader causes an incomplete type errorARROW-250 - Fix for ARROW-246 may cause memory leaksARROW-259 - Use flatbuffer fields in java implementationARROW-260 - TestValueVector.testFixedVectorReallocation and testVariableVectorReallocation are flakyARROW-265 - Negative decimal values have wrong paddingARROW-265 - Negative decimal values have wrong paddingARROW-266 - [C++] Fix the broken buildARROW-274 - Make the MapVector nullableARROW-277 - Flatbuf serialization fails for Timestamp typeARROW-278 - [Format] Struct type name consistency in implementations and metadataARROW-283 - [C++] Update arrow_parquet to account for API changes in PARQUET-573ARROW-284 - [C++] Triage builds by disabling Arrow-Parquet moduleARROW-287 - [java] Make nullable vectors use a BitVecor instead of UInt1Vector for bitsARROW-297 - Fix Arrow pom for releaseARROW-304 - NullableMapReaderImpl.isSet() always returns trueARROW-308 - UnionListWriter.setPosition() should not call startList()ARROW-309 - Types.getMinorTypeForArrowType() does not work for Union typeARROW-313 - XCode 8.0 breaks buildsARROW-314 - JSONScalar is unnecessary and unused.ARROW-320 - ComplexCopier.copy(FieldReader, FieldWriter) should not start a list if reader is not setARROW-321 - Fix Arrow licencesARROW-855 - Arrow Memory Leak
