README.rst 48 KB

12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466
  1. nghttp2 - HTTP/2 C Library
  2. ==========================
  3. This is an implementation of the Hypertext Transfer Protocol version 2
  4. in C.
  5. The framing layer of HTTP/2 is implemented as a reusable C library.
  6. On top of that, we have implemented an HTTP/2 client, server and
  7. proxy. We have also developed load test and benchmarking tools for
  8. HTTP/2.
  9. An HPACK encoder and decoder are available as a public API.
  10. Development Status
  11. ------------------
  12. nghttp2 was originally developed based on `RFC 7540
  13. <https://tools.ietf.org/html/rfc7540>`_ HTTP/2 and `RFC 7541
  14. <https://tools.ietf.org/html/rfc7541>`_ HPACK - Header Compression for
  15. HTTP/2. Now we are updating our code to implement `RFC 9113
  16. <https://datatracker.ietf.org/doc/html/rfc9113>`_.
  17. The nghttp2 code base was forked from the spdylay
  18. (https://github.com/tatsuhiro-t/spdylay) project.
  19. Public Test Server
  20. ------------------
  21. The following endpoints are available to try out our nghttp2
  22. implementation.
  23. * https://nghttp2.org/ (TLS + ALPN and HTTP/3)
  24. This endpoint supports ``h2``, ``h2-16``, ``h2-14``, and
  25. ``http/1.1`` via ALPN and requires TLSv1.2 for HTTP/2
  26. connection.
  27. It also supports HTTP/3.
  28. * http://nghttp2.org/ (HTTP Upgrade and HTTP/2 Direct)
  29. ``h2c`` and ``http/1.1``.
  30. Requirements
  31. ------------
  32. The following package is required to build the libnghttp2 library:
  33. * pkg-config >= 0.20
  34. To build the documentation, you need to install:
  35. * sphinx (http://sphinx-doc.org/)
  36. If you need libnghttp2 (C library) only, then the above packages are
  37. all you need. Use ``--enable-lib-only`` to ensure that only
  38. libnghttp2 is built. This avoids potential build error related to
  39. building bundled applications.
  40. To build and run the application programs (``nghttp``, ``nghttpd``,
  41. ``nghttpx`` and ``h2load``) in the ``src`` directory, the following packages
  42. are required:
  43. * OpenSSL >= 1.1.1; or wolfSSL >= 5.7.0; or LibreSSL >= 3.8.1; or
  44. aws-lc >= 1.19.0; or BoringSSL
  45. * libev >= 4.11
  46. * zlib >= 1.2.3
  47. * libc-ares >= 1.7.5
  48. To enable ``-a`` option (getting linked assets from the downloaded
  49. resource) in ``nghttp``, the following package is required:
  50. * libxml2 >= 2.6.26
  51. To enable systemd support in nghttpx, the following package is
  52. required:
  53. * libsystemd-dev >= 209
  54. The HPACK tools require the following package:
  55. * jansson >= 2.5
  56. To build sources under the examples directory, libevent is required:
  57. * libevent-openssl >= 2.0.8
  58. To mitigate heap fragmentation in long running server programs
  59. (``nghttpd`` and ``nghttpx``), jemalloc is recommended:
  60. * jemalloc
  61. .. note::
  62. Alpine Linux currently does not support malloc replacement
  63. due to musl limitations. See details in issue `#762 <https://github.com/nghttp2/nghttp2/issues/762>`_.
  64. For BoringSSL or aws-lc build, to enable :rfc:`8879` TLS Certificate
  65. Compression in applications, the following library is required:
  66. * libbrotli-dev >= 1.0.9
  67. To enable mruby support for nghttpx, `mruby
  68. <https://github.com/mruby/mruby>`_ is required. We need to build
  69. mruby with C++ ABI explicitly turned on, and probably need other
  70. mrgems, mruby is managed by git submodule under third-party/mruby
  71. directory. Currently, mruby support for nghttpx is disabled by
  72. default. To enable mruby support, use ``--with-mruby`` configure
  73. option. Note that at the time of this writing, libmruby-dev and mruby
  74. packages in Debian/Ubuntu are not usable for nghttp2, since they do
  75. not enable C++ ABI. To build mruby, the following packages are
  76. required:
  77. * ruby
  78. * bison
  79. nghttpx supports `neverbleed <https://github.com/h2o/neverbleed>`_,
  80. privilege separation engine for OpenSSL. In short, it minimizes the
  81. risk of private key leakage when serious bug like Heartbleed is
  82. exploited. The neverbleed is disabled by default. To enable it, use
  83. ``--with-neverbleed`` configure option.
  84. To enable the experimental HTTP/3 support for h2load and nghttpx, the
  85. following libraries are required:
  86. * `OpenSSL with QUIC support
  87. <https://github.com/quictls/openssl/tree/OpenSSL_1_1_1w+quic>`_; or
  88. wolfSSL; or LibreSSL (does not support 0RTT); or aws-lc; or
  89. `BoringSSL <https://boringssl.googlesource.com/boringssl/>`_ (commit
  90. 76968bb3d53982560bcf08bcd0ba3e1865fe15cd)
  91. * `ngtcp2 <https://github.com/ngtcp2/ngtcp2>`_ >= 1.4.0
  92. * `nghttp3 <https://github.com/ngtcp2/nghttp3>`_ >= 1.1.0
  93. Use ``--enable-http3`` configure option to enable HTTP/3 feature for
  94. h2load and nghttpx.
  95. In order to build optional eBPF program to direct an incoming QUIC UDP
  96. datagram to a correct socket for nghttpx, the following libraries are
  97. required:
  98. * libbpf-dev >= 0.7.0
  99. Use ``--with-libbpf`` configure option to build eBPF program.
  100. libelf-dev is needed to build libbpf.
  101. For Ubuntu 20.04, you can build libbpf from `the source code
  102. <https://github.com/libbpf/libbpf/releases>`_. nghttpx requires eBPF
  103. program for reloading its configuration and hot swapping its
  104. executable.
  105. Compiling libnghttp2 C source code requires a C99 compiler. gcc 4.8
  106. is known to be adequate. In order to compile the C++ source code,
  107. C++20 compliant compiler is required. At least g++ >= 12 and
  108. clang++ >= 15 are known to work.
  109. .. note::
  110. To enable mruby support in nghttpx, and use ``--with-mruby``
  111. configure option.
  112. .. note::
  113. Mac OS X users may need the ``--disable-threads`` configure option to
  114. disable multi-threading in nghttpd, nghttpx and h2load to prevent
  115. them from crashing. A patch is welcome to make multi threading work
  116. on Mac OS X platform.
  117. .. note::
  118. To compile the associated applications (nghttp, nghttpd, nghttpx
  119. and h2load), you must use the ``--enable-app`` configure option and
  120. ensure that the specified requirements above are met. Normally,
  121. configure script checks required dependencies to build these
  122. applications, and enable ``--enable-app`` automatically, so you
  123. don't have to use it explicitly. But if you found that
  124. applications were not built, then using ``--enable-app`` may find
  125. that cause, such as the missing dependency.
  126. .. note::
  127. In order to detect third party libraries, pkg-config is used
  128. (however we don't use pkg-config for some libraries (e.g., libev)).
  129. By default, pkg-config searches ``*.pc`` file in the standard
  130. locations (e.g., /usr/lib/pkgconfig). If it is necessary to use
  131. ``*.pc`` file in the custom location, specify paths to
  132. ``PKG_CONFIG_PATH`` environment variable, and pass it to configure
  133. script, like so:
  134. .. code-block:: text
  135. $ ./configure PKG_CONFIG_PATH=/path/to/pkgconfig
  136. For pkg-config managed libraries, ``*_CFLAG`` and ``*_LIBS``
  137. environment variables are defined (e.g., ``OPENSSL_CFLAGS``,
  138. ``OPENSSL_LIBS``). Specifying non-empty string to these variables
  139. completely overrides pkg-config. In other words, if they are
  140. specified, pkg-config is not used for detection, and user is
  141. responsible to specify the correct values to these variables. For
  142. complete list of these variables, run ``./configure -h``.
  143. If you are using Ubuntu 22.04 LTS, run the following to install the
  144. required packages:
  145. .. code-block:: text
  146. sudo apt-get install g++ clang make binutils autoconf automake \
  147. autotools-dev libtool pkg-config \
  148. zlib1g-dev libssl-dev libxml2-dev libev-dev \
  149. libevent-dev libjansson-dev \
  150. libc-ares-dev libjemalloc-dev libsystemd-dev \
  151. ruby-dev bison libelf-dev
  152. Building nghttp2 from release tar archive
  153. -----------------------------------------
  154. The nghttp2 project regularly releases tar archives which includes
  155. nghttp2 source code, and generated build files. They can be
  156. downloaded from `Releases
  157. <https://github.com/nghttp2/nghttp2/releases>`_ page.
  158. Building nghttp2 from git requires autotools development packages.
  159. Building from tar archives does not require them, and thus it is much
  160. easier. The usual build step is as follows:
  161. .. code-block:: text
  162. $ tar xf nghttp2-X.Y.Z.tar.bz2
  163. $ cd nghttp2-X.Y.Z
  164. $ ./configure
  165. $ make
  166. Building from git
  167. -----------------
  168. Building from git is easy, but please be sure that at least autoconf 2.68 is
  169. used:
  170. .. code-block:: text
  171. $ git submodule update --init
  172. $ autoreconf -i
  173. $ automake
  174. $ autoconf
  175. $ ./configure
  176. $ make
  177. Notes for building on Windows (MSVC)
  178. ------------------------------------
  179. The easiest way to build native Windows nghttp2 dll is use `cmake
  180. <https://cmake.org/>`_. The free version of `Visual C++ Build Tools
  181. <http://landinghub.visualstudio.com/visual-cpp-build-tools>`_ works
  182. fine.
  183. 1. Install cmake for windows
  184. 2. Open "Visual C++ ... Native Build Tool Command Prompt", and inside
  185. nghttp2 directly, run ``cmake``.
  186. 3. Then run ``cmake --build`` to build library.
  187. 4. nghttp2.dll, nghttp2.lib, nghttp2.exp are placed under lib directory.
  188. Note that the above steps most likely produce nghttp2 library only.
  189. No bundled applications are compiled.
  190. Notes for building on Windows (Mingw/Cygwin)
  191. --------------------------------------------
  192. Under Mingw environment, you can only compile the library, it's
  193. ``libnghttp2-X.dll`` and ``libnghttp2.a``.
  194. If you want to compile the applications(``h2load``, ``nghttp``,
  195. ``nghttpx``, ``nghttpd``), you need to use the Cygwin environment.
  196. Under Cygwin environment, to compile the applications you need to
  197. compile and install the libev first.
  198. Secondly, you need to undefine the macro ``__STRICT_ANSI__``, if you
  199. not, the functions ``fdopen``, ``fileno`` and ``strptime`` will not
  200. available.
  201. the sample command like this:
  202. .. code-block:: text
  203. $ export CFLAGS="-U__STRICT_ANSI__ -I$libev_PREFIX/include -L$libev_PREFIX/lib"
  204. $ export CXXFLAGS=$CFLAGS
  205. $ ./configure
  206. $ make
  207. If you want to compile the applications under ``examples/``, you need
  208. to remove or rename the ``event.h`` from libev's installation, because
  209. it conflicts with libevent's installation.
  210. Notes for installation on Linux systems
  211. --------------------------------------------
  212. After installing nghttp2 tool suite with ``make install`` one might experience a similar error:
  213. .. code-block:: text
  214. nghttpx: error while loading shared libraries: libnghttp2.so.14: cannot open shared object file: No such file or directory
  215. This means that the tool is unable to locate the ``libnghttp2.so`` shared library.
  216. To update the shared library cache run ``sudo ldconfig``.
  217. Building the documentation
  218. --------------------------
  219. .. note::
  220. Documentation is still incomplete.
  221. To build the documentation, run:
  222. .. code-block:: text
  223. $ make html
  224. The documents will be generated under ``doc/manual/html/``.
  225. The generated documents will not be installed with ``make install``.
  226. The online documentation is available at
  227. https://nghttp2.org/documentation/
  228. Build HTTP/3 enabled h2load and nghttpx
  229. ---------------------------------------
  230. To build h2load and nghttpx with HTTP/3 feature enabled, run the
  231. configure script with ``--enable-http3``.
  232. For nghttpx to reload configurations and swapping its executable while
  233. gracefully terminating old worker processes, eBPF is required. Run
  234. the configure script with ``--enable-http3 --with-libbpf`` to build
  235. eBPF program. The QUIC keying material must be set with
  236. ``--frontend-quic-secret-file`` in order to keep the existing
  237. connections alive during reload.
  238. The detailed steps to build HTTP/3 enabled h2load and nghttpx follow.
  239. Build aws-lc:
  240. .. code-block:: text
  241. $ git clone --depth 1 -b v1.36.1 https://github.com/aws/aws-lc
  242. $ cd aws-lc
  243. $ cmake -B build -DDISABLE_GO=ON --install-prefix=$PWD/opt
  244. $ make -j$(nproc) -C build
  245. $ cmake --install build
  246. $ cd ..
  247. Build nghttp3:
  248. .. code-block:: text
  249. $ git clone --depth 1 -b v1.6.0 https://github.com/ngtcp2/nghttp3
  250. $ cd nghttp3
  251. $ git submodule update --init --depth 1
  252. $ autoreconf -i
  253. $ ./configure --prefix=$PWD/build --enable-lib-only
  254. $ make -j$(nproc)
  255. $ make install
  256. $ cd ..
  257. Build ngtcp2:
  258. .. code-block:: text
  259. $ git clone --depth 1 -b v1.8.1 https://github.com/ngtcp2/ngtcp2
  260. $ cd ngtcp2
  261. $ git submodule update --init --depth 1
  262. $ autoreconf -i
  263. $ ./configure --prefix=$PWD/build --enable-lib-only --with-boringssl \
  264. BORINGSSL_CFLAGS="-I$PWD/../aws-lc/opt/include" \
  265. BORINGSSL_LIBS="-L$PWD/../aws-lc/opt/lib -lssl -lcrypto"
  266. $ make -j$(nproc)
  267. $ make install
  268. $ cd ..
  269. If your Linux distribution does not have libbpf-dev >= 0.7.0, build
  270. from source:
  271. .. code-block:: text
  272. $ git clone --depth 1 -b v1.4.6 https://github.com/libbpf/libbpf
  273. $ cd libbpf
  274. $ PREFIX=$PWD/build make -C src install
  275. $ cd ..
  276. Build nghttp2:
  277. .. code-block:: text
  278. $ git clone https://github.com/nghttp2/nghttp2
  279. $ cd nghttp2
  280. $ git submodule update --init
  281. $ autoreconf -i
  282. $ ./configure --with-mruby --enable-http3 --with-libbpf \
  283. CC=clang-15 CXX=clang++-15 \
  284. PKG_CONFIG_PATH="$PWD/../aws-lc/opt/lib/pkgconfig:$PWD/../nghttp3/build/lib/pkgconfig:$PWD/../ngtcp2/build/lib/pkgconfig:$PWD/../libbpf/build/lib64/pkgconfig" \
  285. LDFLAGS="$LDFLAGS -Wl,-rpath,$PWD/../aws-lc/opt/lib -Wl,-rpath,$PWD/../libbpf/build/lib64"
  286. $ make -j$(nproc)
  287. The eBPF program ``reuseport_kern.o`` should be found under bpf
  288. directory. Pass ``--quic-bpf-program-file=bpf/reuseport_kern.o``
  289. option to nghttpx to load it. See also `HTTP/3 section in nghttpx -
  290. HTTP/2 proxy - HOW-TO
  291. <https://nghttp2.org/documentation/nghttpx-howto.html#http-3>`_.
  292. Unit tests
  293. ----------
  294. Unit tests are done by simply running ``make check``.
  295. Integration tests
  296. -----------------
  297. We have the integration tests for the nghttpx proxy server. The tests are
  298. written in the `Go programming language <http://golang.org/>`_ and uses
  299. its testing framework. We depend on the following libraries:
  300. * golang.org/x/net/http2
  301. * golang.org/x/net/websocket
  302. * https://github.com/tatsuhiro-t/go-nghttp2
  303. Go modules will download these dependencies automatically.
  304. To run the tests, run the following command under
  305. ``integration-tests`` directory:
  306. .. code-block:: text
  307. $ make it
  308. Inside the tests, we use port 3009 to run the test subject server.
  309. Migration from v0.7.15 or earlier
  310. ---------------------------------
  311. nghttp2 v1.0.0 introduced several backward incompatible changes. In
  312. this section, we describe these changes and how to migrate to v1.0.0.
  313. ALPN protocol ID is now ``h2`` and ``h2c``
  314. ++++++++++++++++++++++++++++++++++++++++++
  315. Previously we announced ``h2-14`` and ``h2c-14``. v1.0.0 implements
  316. final protocol version, and we changed ALPN ID to ``h2`` and ``h2c``.
  317. The macros ``NGHTTP2_PROTO_VERSION_ID``,
  318. ``NGHTTP2_PROTO_VERSION_ID_LEN``,
  319. ``NGHTTP2_CLEARTEXT_PROTO_VERSION_ID``, and
  320. ``NGHTTP2_CLEARTEXT_PROTO_VERSION_ID_LEN`` have been updated to
  321. reflect this change.
  322. Basically, existing applications do not have to do anything, just
  323. recompiling is enough for this change.
  324. Use word "client magic" where we use "client connection preface"
  325. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  326. We use "client connection preface" to mean first 24 bytes of client
  327. connection preface. This is technically not correct, since client
  328. connection preface is composed of 24 bytes client magic byte string
  329. followed by SETTINGS frame. For clarification, we call "client magic"
  330. for this 24 bytes byte string and updated API.
  331. * ``NGHTTP2_CLIENT_CONNECTION_PREFACE`` was replaced with
  332. ``NGHTTP2_CLIENT_MAGIC``.
  333. * ``NGHTTP2_CLIENT_CONNECTION_PREFACE_LEN`` was replaced with
  334. ``NGHTTP2_CLIENT_MAGIC_LEN``.
  335. * ``NGHTTP2_BAD_PREFACE`` was renamed as ``NGHTTP2_BAD_CLIENT_MAGIC``
  336. The already deprecated ``NGHTTP2_CLIENT_CONNECTION_HEADER`` and
  337. ``NGHTTP2_CLIENT_CONNECTION_HEADER_LEN`` were removed.
  338. If application uses these macros, just replace old ones with new ones.
  339. Since v1.0.0, client magic is sent by library (see next subsection),
  340. so client application may just remove these macro use.
  341. Client magic is sent by library
  342. +++++++++++++++++++++++++++++++
  343. Previously nghttp2 library did not send client magic, which is first
  344. 24 bytes byte string of client connection preface, and client
  345. applications have to send it by themselves. Since v1.0.0, client
  346. magic is sent by library via first call of ``nghttp2_session_send()``
  347. or ``nghttp2_session_mem_send2()``.
  348. The client applications which send client magic must remove the
  349. relevant code.
  350. Remove HTTP Alternative Services (Alt-Svc) related code
  351. +++++++++++++++++++++++++++++++++++++++++++++++++++++++
  352. Alt-Svc specification is not finalized yet. To make our API stable,
  353. we have decided to remove all Alt-Svc related API from nghttp2.
  354. * ``NGHTTP2_EXT_ALTSVC`` was removed.
  355. * ``nghttp2_ext_altsvc`` was removed.
  356. We have already removed the functionality of Alt-Svc in v0.7 series
  357. and they have been essentially noop. The application using these
  358. macro and struct, remove those lines.
  359. Use nghttp2_error in nghttp2_on_invalid_frame_recv_callback
  360. +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  361. Previously ``nghttp2_on_invalid_frame_recv_cb_called`` took the
  362. ``error_code``, defined in ``nghttp2_error_code``, as parameter. But
  363. they are not detailed enough to debug. Therefore, we decided to use
  364. more detailed ``nghttp2_error`` values instead.
  365. The application using this callback should update the callback
  366. signature. If it treats ``error_code`` as HTTP/2 error code, update
  367. the code so that it is treated as ``nghttp2_error``.
  368. Receive client magic by default
  369. +++++++++++++++++++++++++++++++
  370. Previously nghttp2 did not process client magic (24 bytes byte
  371. string). To make it deal with it, we had to use
  372. ``nghttp2_option_set_recv_client_preface()``. Since v1.0.0, nghttp2
  373. processes client magic by default and
  374. ``nghttp2_option_set_recv_client_preface()`` was removed.
  375. Some application may want to disable this behaviour, so we added
  376. ``nghttp2_option_set_no_recv_client_magic()`` to achieve this.
  377. The application using ``nghttp2_option_set_recv_client_preface()``
  378. with nonzero value, just remove it.
  379. The application using ``nghttp2_option_set_recv_client_preface()``
  380. with zero value or not using it must use
  381. ``nghttp2_option_set_no_recv_client_magic()`` with nonzero value.
  382. Client, Server and Proxy programs
  383. ---------------------------------
  384. The ``src`` directory contains the HTTP/2 client, server and proxy programs.
  385. nghttp - client
  386. +++++++++++++++
  387. ``nghttp`` is a HTTP/2 client. It can connect to the HTTP/2 server
  388. with prior knowledge, HTTP Upgrade and ALPN TLS extension.
  389. It has verbose output mode for framing information. Here is sample
  390. output from ``nghttp`` client:
  391. .. code-block:: text
  392. $ nghttp -nv https://nghttp2.org
  393. [ 0.190] Connected
  394. The negotiated protocol: h2
  395. [ 0.212] recv SETTINGS frame <length=12, flags=0x00, stream_id=0>
  396. (niv=2)
  397. [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
  398. [SETTINGS_INITIAL_WINDOW_SIZE(0x04):65535]
  399. [ 0.212] send SETTINGS frame <length=12, flags=0x00, stream_id=0>
  400. (niv=2)
  401. [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
  402. [SETTINGS_INITIAL_WINDOW_SIZE(0x04):65535]
  403. [ 0.212] send SETTINGS frame <length=0, flags=0x01, stream_id=0>
  404. ; ACK
  405. (niv=0)
  406. [ 0.212] send PRIORITY frame <length=5, flags=0x00, stream_id=3>
  407. (dep_stream_id=0, weight=201, exclusive=0)
  408. [ 0.212] send PRIORITY frame <length=5, flags=0x00, stream_id=5>
  409. (dep_stream_id=0, weight=101, exclusive=0)
  410. [ 0.212] send PRIORITY frame <length=5, flags=0x00, stream_id=7>
  411. (dep_stream_id=0, weight=1, exclusive=0)
  412. [ 0.212] send PRIORITY frame <length=5, flags=0x00, stream_id=9>
  413. (dep_stream_id=7, weight=1, exclusive=0)
  414. [ 0.212] send PRIORITY frame <length=5, flags=0x00, stream_id=11>
  415. (dep_stream_id=3, weight=1, exclusive=0)
  416. [ 0.212] send HEADERS frame <length=39, flags=0x25, stream_id=13>
  417. ; END_STREAM | END_HEADERS | PRIORITY
  418. (padlen=0, dep_stream_id=11, weight=16, exclusive=0)
  419. ; Open new stream
  420. :method: GET
  421. :path: /
  422. :scheme: https
  423. :authority: nghttp2.org
  424. accept: */*
  425. accept-encoding: gzip, deflate
  426. user-agent: nghttp2/1.0.1-DEV
  427. [ 0.221] recv SETTINGS frame <length=0, flags=0x01, stream_id=0>
  428. ; ACK
  429. (niv=0)
  430. [ 0.221] recv (stream_id=13) :method: GET
  431. [ 0.221] recv (stream_id=13) :scheme: https
  432. [ 0.221] recv (stream_id=13) :path: /stylesheets/screen.css
  433. [ 0.221] recv (stream_id=13) :authority: nghttp2.org
  434. [ 0.221] recv (stream_id=13) accept-encoding: gzip, deflate
  435. [ 0.222] recv (stream_id=13) user-agent: nghttp2/1.0.1-DEV
  436. [ 0.222] recv PUSH_PROMISE frame <length=50, flags=0x04, stream_id=13>
  437. ; END_HEADERS
  438. (padlen=0, promised_stream_id=2)
  439. [ 0.222] recv (stream_id=13) :status: 200
  440. [ 0.222] recv (stream_id=13) date: Thu, 21 May 2015 16:38:14 GMT
  441. [ 0.222] recv (stream_id=13) content-type: text/html
  442. [ 0.222] recv (stream_id=13) last-modified: Fri, 15 May 2015 15:38:06 GMT
  443. [ 0.222] recv (stream_id=13) etag: W/"555612de-19f6"
  444. [ 0.222] recv (stream_id=13) link: </stylesheets/screen.css>; rel=preload; as=stylesheet
  445. [ 0.222] recv (stream_id=13) content-encoding: gzip
  446. [ 0.222] recv (stream_id=13) server: nghttpx nghttp2/1.0.1-DEV
  447. [ 0.222] recv (stream_id=13) via: 1.1 nghttpx
  448. [ 0.222] recv (stream_id=13) strict-transport-security: max-age=31536000
  449. [ 0.222] recv HEADERS frame <length=166, flags=0x04, stream_id=13>
  450. ; END_HEADERS
  451. (padlen=0)
  452. ; First response header
  453. [ 0.222] recv DATA frame <length=2601, flags=0x01, stream_id=13>
  454. ; END_STREAM
  455. [ 0.222] recv (stream_id=2) :status: 200
  456. [ 0.222] recv (stream_id=2) date: Thu, 21 May 2015 16:38:14 GMT
  457. [ 0.222] recv (stream_id=2) content-type: text/css
  458. [ 0.222] recv (stream_id=2) last-modified: Fri, 15 May 2015 15:38:06 GMT
  459. [ 0.222] recv (stream_id=2) etag: W/"555612de-9845"
  460. [ 0.222] recv (stream_id=2) content-encoding: gzip
  461. [ 0.222] recv (stream_id=2) server: nghttpx nghttp2/1.0.1-DEV
  462. [ 0.222] recv (stream_id=2) via: 1.1 nghttpx
  463. [ 0.222] recv (stream_id=2) strict-transport-security: max-age=31536000
  464. [ 0.222] recv HEADERS frame <length=32, flags=0x04, stream_id=2>
  465. ; END_HEADERS
  466. (padlen=0)
  467. ; First push response header
  468. [ 0.228] recv DATA frame <length=8715, flags=0x01, stream_id=2>
  469. ; END_STREAM
  470. [ 0.228] send GOAWAY frame <length=8, flags=0x00, stream_id=0>
  471. (last_stream_id=2, error_code=NO_ERROR(0x00), opaque_data(0)=[])
  472. The HTTP Upgrade is performed like so:
  473. .. code-block:: text
  474. $ nghttp -nvu http://nghttp2.org
  475. [ 0.011] Connected
  476. [ 0.011] HTTP Upgrade request
  477. GET / HTTP/1.1
  478. Host: nghttp2.org
  479. Connection: Upgrade, HTTP2-Settings
  480. Upgrade: h2c
  481. HTTP2-Settings: AAMAAABkAAQAAP__
  482. Accept: */*
  483. User-Agent: nghttp2/1.0.1-DEV
  484. [ 0.018] HTTP Upgrade response
  485. HTTP/1.1 101 Switching Protocols
  486. Connection: Upgrade
  487. Upgrade: h2c
  488. [ 0.018] HTTP Upgrade success
  489. [ 0.018] recv SETTINGS frame <length=12, flags=0x00, stream_id=0>
  490. (niv=2)
  491. [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
  492. [SETTINGS_INITIAL_WINDOW_SIZE(0x04):65535]
  493. [ 0.018] send SETTINGS frame <length=12, flags=0x00, stream_id=0>
  494. (niv=2)
  495. [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
  496. [SETTINGS_INITIAL_WINDOW_SIZE(0x04):65535]
  497. [ 0.018] send SETTINGS frame <length=0, flags=0x01, stream_id=0>
  498. ; ACK
  499. (niv=0)
  500. [ 0.018] send PRIORITY frame <length=5, flags=0x00, stream_id=3>
  501. (dep_stream_id=0, weight=201, exclusive=0)
  502. [ 0.018] send PRIORITY frame <length=5, flags=0x00, stream_id=5>
  503. (dep_stream_id=0, weight=101, exclusive=0)
  504. [ 0.018] send PRIORITY frame <length=5, flags=0x00, stream_id=7>
  505. (dep_stream_id=0, weight=1, exclusive=0)
  506. [ 0.018] send PRIORITY frame <length=5, flags=0x00, stream_id=9>
  507. (dep_stream_id=7, weight=1, exclusive=0)
  508. [ 0.018] send PRIORITY frame <length=5, flags=0x00, stream_id=11>
  509. (dep_stream_id=3, weight=1, exclusive=0)
  510. [ 0.018] send PRIORITY frame <length=5, flags=0x00, stream_id=1>
  511. (dep_stream_id=11, weight=16, exclusive=0)
  512. [ 0.019] recv (stream_id=1) :method: GET
  513. [ 0.019] recv (stream_id=1) :scheme: http
  514. [ 0.019] recv (stream_id=1) :path: /stylesheets/screen.css
  515. [ 0.019] recv (stream_id=1) host: nghttp2.org
  516. [ 0.019] recv (stream_id=1) user-agent: nghttp2/1.0.1-DEV
  517. [ 0.019] recv PUSH_PROMISE frame <length=49, flags=0x04, stream_id=1>
  518. ; END_HEADERS
  519. (padlen=0, promised_stream_id=2)
  520. [ 0.019] recv (stream_id=1) :status: 200
  521. [ 0.019] recv (stream_id=1) date: Thu, 21 May 2015 16:39:16 GMT
  522. [ 0.019] recv (stream_id=1) content-type: text/html
  523. [ 0.019] recv (stream_id=1) content-length: 6646
  524. [ 0.019] recv (stream_id=1) last-modified: Fri, 15 May 2015 15:38:06 GMT
  525. [ 0.019] recv (stream_id=1) etag: "555612de-19f6"
  526. [ 0.019] recv (stream_id=1) link: </stylesheets/screen.css>; rel=preload; as=stylesheet
  527. [ 0.019] recv (stream_id=1) accept-ranges: bytes
  528. [ 0.019] recv (stream_id=1) server: nghttpx nghttp2/1.0.1-DEV
  529. [ 0.019] recv (stream_id=1) via: 1.1 nghttpx
  530. [ 0.019] recv HEADERS frame <length=157, flags=0x04, stream_id=1>
  531. ; END_HEADERS
  532. (padlen=0)
  533. ; First response header
  534. [ 0.019] recv DATA frame <length=6646, flags=0x01, stream_id=1>
  535. ; END_STREAM
  536. [ 0.019] recv (stream_id=2) :status: 200
  537. [ 0.019] recv (stream_id=2) date: Thu, 21 May 2015 16:39:16 GMT
  538. [ 0.019] recv (stream_id=2) content-type: text/css
  539. [ 0.019] recv (stream_id=2) content-length: 38981
  540. [ 0.019] recv (stream_id=2) last-modified: Fri, 15 May 2015 15:38:06 GMT
  541. [ 0.019] recv (stream_id=2) etag: "555612de-9845"
  542. [ 0.019] recv (stream_id=2) accept-ranges: bytes
  543. [ 0.019] recv (stream_id=2) server: nghttpx nghttp2/1.0.1-DEV
  544. [ 0.019] recv (stream_id=2) via: 1.1 nghttpx
  545. [ 0.019] recv HEADERS frame <length=36, flags=0x04, stream_id=2>
  546. ; END_HEADERS
  547. (padlen=0)
  548. ; First push response header
  549. [ 0.026] recv DATA frame <length=16384, flags=0x00, stream_id=2>
  550. [ 0.027] recv DATA frame <length=7952, flags=0x00, stream_id=2>
  551. [ 0.027] send WINDOW_UPDATE frame <length=4, flags=0x00, stream_id=0>
  552. (window_size_increment=33343)
  553. [ 0.032] send WINDOW_UPDATE frame <length=4, flags=0x00, stream_id=2>
  554. (window_size_increment=33707)
  555. [ 0.032] recv DATA frame <length=14645, flags=0x01, stream_id=2>
  556. ; END_STREAM
  557. [ 0.032] recv SETTINGS frame <length=0, flags=0x01, stream_id=0>
  558. ; ACK
  559. (niv=0)
  560. [ 0.032] send GOAWAY frame <length=8, flags=0x00, stream_id=0>
  561. (last_stream_id=2, error_code=NO_ERROR(0x00), opaque_data(0)=[])
  562. Using the ``-s`` option, ``nghttp`` prints out some timing information for
  563. requests, sorted by completion time:
  564. .. code-block:: text
  565. $ nghttp -nas https://nghttp2.org/
  566. ***** Statistics *****
  567. Request timing:
  568. responseEnd: the time when last byte of response was received
  569. relative to connectEnd
  570. requestStart: the time just before first byte of request was sent
  571. relative to connectEnd. If '*' is shown, this was
  572. pushed by server.
  573. process: responseEnd - requestStart
  574. code: HTTP status code
  575. size: number of bytes received as response body without
  576. inflation.
  577. URI: request URI
  578. see http://www.w3.org/TR/resource-timing/#processing-model
  579. sorted by 'complete'
  580. id responseEnd requestStart process code size request path
  581. 13 +37.19ms +280us 36.91ms 200 2K /
  582. 2 +72.65ms * +36.38ms 36.26ms 200 8K /stylesheets/screen.css
  583. 17 +77.43ms +38.67ms 38.75ms 200 3K /javascripts/octopress.js
  584. 15 +78.12ms +38.66ms 39.46ms 200 3K /javascripts/modernizr-2.0.js
  585. Using the ``-r`` option, ``nghttp`` writes more detailed timing data to
  586. the given file in HAR format.
  587. nghttpd - server
  588. ++++++++++++++++
  589. ``nghttpd`` is a multi-threaded static web server.
  590. By default, it uses SSL/TLS connection. Use ``--no-tls`` option to
  591. disable it.
  592. ``nghttpd`` only accepts HTTP/2 connections via ALPN or direct HTTP/2
  593. connections. No HTTP Upgrade is supported.
  594. The ``-p`` option allows users to configure server push.
  595. Just like ``nghttp``, it has a verbose output mode for framing
  596. information. Here is sample output from ``nghttpd``:
  597. .. code-block:: text
  598. $ nghttpd --no-tls -v 8080
  599. IPv4: listen 0.0.0.0:8080
  600. IPv6: listen :::8080
  601. [id=1] [ 1.521] send SETTINGS frame <length=6, flags=0x00, stream_id=0>
  602. (niv=1)
  603. [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
  604. [id=1] [ 1.521] recv SETTINGS frame <length=12, flags=0x00, stream_id=0>
  605. (niv=2)
  606. [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
  607. [SETTINGS_INITIAL_WINDOW_SIZE(0x04):65535]
  608. [id=1] [ 1.521] recv SETTINGS frame <length=0, flags=0x01, stream_id=0>
  609. ; ACK
  610. (niv=0)
  611. [id=1] [ 1.521] recv PRIORITY frame <length=5, flags=0x00, stream_id=3>
  612. (dep_stream_id=0, weight=201, exclusive=0)
  613. [id=1] [ 1.521] recv PRIORITY frame <length=5, flags=0x00, stream_id=5>
  614. (dep_stream_id=0, weight=101, exclusive=0)
  615. [id=1] [ 1.521] recv PRIORITY frame <length=5, flags=0x00, stream_id=7>
  616. (dep_stream_id=0, weight=1, exclusive=0)
  617. [id=1] [ 1.521] recv PRIORITY frame <length=5, flags=0x00, stream_id=9>
  618. (dep_stream_id=7, weight=1, exclusive=0)
  619. [id=1] [ 1.521] recv PRIORITY frame <length=5, flags=0x00, stream_id=11>
  620. (dep_stream_id=3, weight=1, exclusive=0)
  621. [id=1] [ 1.521] recv (stream_id=13) :method: GET
  622. [id=1] [ 1.521] recv (stream_id=13) :path: /
  623. [id=1] [ 1.521] recv (stream_id=13) :scheme: http
  624. [id=1] [ 1.521] recv (stream_id=13) :authority: localhost:8080
  625. [id=1] [ 1.521] recv (stream_id=13) accept: */*
  626. [id=1] [ 1.521] recv (stream_id=13) accept-encoding: gzip, deflate
  627. [id=1] [ 1.521] recv (stream_id=13) user-agent: nghttp2/1.0.0-DEV
  628. [id=1] [ 1.521] recv HEADERS frame <length=41, flags=0x25, stream_id=13>
  629. ; END_STREAM | END_HEADERS | PRIORITY
  630. (padlen=0, dep_stream_id=11, weight=16, exclusive=0)
  631. ; Open new stream
  632. [id=1] [ 1.521] send SETTINGS frame <length=0, flags=0x01, stream_id=0>
  633. ; ACK
  634. (niv=0)
  635. [id=1] [ 1.521] send HEADERS frame <length=86, flags=0x04, stream_id=13>
  636. ; END_HEADERS
  637. (padlen=0)
  638. ; First response header
  639. :status: 200
  640. server: nghttpd nghttp2/1.0.0-DEV
  641. content-length: 10
  642. cache-control: max-age=3600
  643. date: Fri, 15 May 2015 14:49:04 GMT
  644. last-modified: Tue, 30 Sep 2014 12:40:52 GMT
  645. [id=1] [ 1.522] send DATA frame <length=10, flags=0x01, stream_id=13>
  646. ; END_STREAM
  647. [id=1] [ 1.522] stream_id=13 closed
  648. [id=1] [ 1.522] recv GOAWAY frame <length=8, flags=0x00, stream_id=0>
  649. (last_stream_id=0, error_code=NO_ERROR(0x00), opaque_data(0)=[])
  650. [id=1] [ 1.522] closed
  651. nghttpx - proxy
  652. +++++++++++++++
  653. ``nghttpx`` is a multi-threaded reverse proxy for HTTP/3, HTTP/2, and
  654. HTTP/1.1, and powers http://nghttp2.org and supports HTTP/2 server
  655. push.
  656. We reworked ``nghttpx`` command-line interface, and as a result, there
  657. are several incompatibles from 1.8.0 or earlier. This is necessary to
  658. extend its capability, and secure the further feature enhancements in
  659. the future release. Please read `Migration from nghttpx v1.8.0 or
  660. earlier
  661. <https://nghttp2.org/documentation/nghttpx-howto.html#migration-from-nghttpx-v1-8-0-or-earlier>`_
  662. to know how to migrate from earlier releases.
  663. ``nghttpx`` implements `important performance-oriented features
  664. <https://istlsfastyet.com/#server-performance>`_ in TLS, such as
  665. session IDs, session tickets (with automatic key rotation), OCSP
  666. stapling, dynamic record sizing, ALPN, forward secrecy and HTTP/2.
  667. ``nghttpx`` also offers the functionality to share session cache and
  668. ticket keys among multiple ``nghttpx`` instances via memcached.
  669. ``nghttpx`` has 2 operation modes:
  670. ================== ======================== ================ =============
  671. Mode option Frontend Backend Note
  672. ================== ======================== ================ =============
  673. default mode HTTP/3, HTTP/2, HTTP/1.1 HTTP/1.1, HTTP/2 Reverse proxy
  674. ``--http2-proxy`` HTTP/3, HTTP/2, HTTP/1.1 HTTP/1.1, HTTP/2 Forward proxy
  675. ================== ======================== ================ =============
  676. The interesting mode at the moment is the default mode. It works like
  677. a reverse proxy and listens for HTTP/3, HTTP/2, and HTTP/1.1 and can
  678. be deployed as a SSL/TLS terminator for existing web server.
  679. In all modes, the frontend connections are encrypted by SSL/TLS by
  680. default. To disable encryption, use the ``no-tls`` keyword in
  681. ``--frontend`` option. If encryption is disabled, incoming HTTP/1.1
  682. connections can be upgraded to HTTP/2 through HTTP Upgrade. On the
  683. other hard, backend connections are not encrypted by default. To
  684. encrypt backend connections, use ``tls`` keyword in ``--backend``
  685. option.
  686. ``nghttpx`` supports a configuration file. See the ``--conf`` option and
  687. sample configuration file ``nghttpx.conf.sample``.
  688. In the default mode, ``nghttpx`` works as reverse proxy to the backend
  689. server:
  690. .. code-block:: text
  691. Client <-- (HTTP/3, HTTP/2, HTTP/1.1) --> nghttpx <-- (HTTP/1.1, HTTP/2) --> Web Server
  692. [reverse proxy]
  693. With the ``--http2-proxy`` option, it works as forward proxy, and it
  694. is so called secure HTTP/2 proxy:
  695. .. code-block:: text
  696. Client <-- (HTTP/3, HTTP/2, HTTP/1.1) --> nghttpx <-- (HTTP/1.1) --> Proxy
  697. [secure proxy] (e.g., Squid, ATS)
  698. The ``Client`` in the above example needs to be configured to use
  699. ``nghttpx`` as secure proxy.
  700. At the time of this writing, both Chrome and Firefox support secure
  701. HTTP/2 proxy. One way to configure Chrome to use a secure proxy is to
  702. create a proxy.pac script like this:
  703. .. code-block:: javascript
  704. function FindProxyForURL(url, host) {
  705. return "HTTPS SERVERADDR:PORT";
  706. }
  707. ``SERVERADDR`` and ``PORT`` is the hostname/address and port of the
  708. machine nghttpx is running on. Please note that Chrome requires a valid
  709. certificate for secure proxy.
  710. Then run Chrome with the following arguments:
  711. .. code-block:: text
  712. $ google-chrome --proxy-pac-url=file:///path/to/proxy.pac --use-npn
  713. The backend HTTP/2 connections can be tunneled through an HTTP proxy.
  714. The proxy is specified using ``--backend-http-proxy-uri``. The
  715. following figure illustrates how nghttpx talks to the outside HTTP/2
  716. proxy through an HTTP proxy:
  717. .. code-block:: text
  718. Client <-- (HTTP/3, HTTP/2, HTTP/1.1) --> nghttpx <-- (HTTP/2) --
  719. --===================---> HTTP/2 Proxy
  720. (HTTP proxy tunnel) (e.g., nghttpx -s)
  721. Benchmarking tool
  722. -----------------
  723. The ``h2load`` program is a benchmarking tool for HTTP/3, HTTP/2, and
  724. HTTP/1.1. The UI of ``h2load`` is heavily inspired by ``weighttp``
  725. (https://github.com/lighttpd/weighttp). The typical usage is as
  726. follows:
  727. .. code-block:: text
  728. $ h2load -n100000 -c100 -m100 https://localhost:8443/
  729. starting benchmark...
  730. spawning thread #0: 100 concurrent clients, 100000 total requests
  731. Protocol: TLSv1.2
  732. Cipher: ECDHE-RSA-AES128-GCM-SHA256
  733. Server Temp Key: ECDH P-256 256 bits
  734. progress: 10% done
  735. progress: 20% done
  736. progress: 30% done
  737. progress: 40% done
  738. progress: 50% done
  739. progress: 60% done
  740. progress: 70% done
  741. progress: 80% done
  742. progress: 90% done
  743. progress: 100% done
  744. finished in 771.26ms, 129658 req/s, 4.71MB/s
  745. requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored
  746. status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
  747. traffic: 3812300 bytes total, 1009900 bytes headers, 1000000 bytes data
  748. min max mean sd +/- sd
  749. time for request: 25.12ms 124.55ms 51.07ms 15.36ms 84.87%
  750. time for connect: 208.94ms 254.67ms 241.38ms 7.95ms 63.00%
  751. time to 1st byte: 209.11ms 254.80ms 241.51ms 7.94ms 63.00%
  752. The above example issued total 100,000 requests, using 100 concurrent
  753. clients (in other words, 100 HTTP/2 sessions), and a maximum of 100 streams
  754. per client. With the ``-t`` option, ``h2load`` will use multiple native
  755. threads to avoid saturating a single core on client side.
  756. .. warning::
  757. **Don't use this tool against publicly available servers.** That is
  758. considered a DOS attack. Please only use it against your private
  759. servers.
  760. If the experimental HTTP/3 is enabled, h2load can send requests to
  761. HTTP/3 server. To do this, specify ``h3`` to ``--alpn-list`` option
  762. like so:
  763. .. code-block:: text
  764. $ h2load --alpn-list h3 https://127.0.0.1:4433
  765. For nghttp2 v1.58 or earlier, use ``--npn-list`` instead of
  766. ``--alpn-list``.
  767. HPACK tools
  768. -----------
  769. The ``src`` directory contains the HPACK tools. The ``deflatehd`` program is a
  770. command-line header compression tool. The ``inflatehd`` program is a
  771. command-line header decompression tool. Both tools read input from
  772. stdin and write output to stdout. Errors are written to stderr.
  773. They take JSON as input and output. We (mostly) use the same JSON data
  774. format described at https://github.com/http2jp/hpack-test-case.
  775. deflatehd - header compressor
  776. +++++++++++++++++++++++++++++
  777. The ``deflatehd`` program reads JSON data or HTTP/1-style header fields from
  778. stdin and outputs compressed header block in JSON.
  779. For the JSON input, the root JSON object must include a ``cases`` key.
  780. Its value has to include the sequence of input header set. They share
  781. the same compression context and are processed in the order they
  782. appear. Each item in the sequence is a JSON object and it must
  783. include a ``headers`` key. Its value is an array of JSON objects,
  784. which includes exactly one name/value pair.
  785. Example:
  786. .. code-block:: json
  787. {
  788. "cases":
  789. [
  790. {
  791. "headers": [
  792. { ":method": "GET" },
  793. { ":path": "/" }
  794. ]
  795. },
  796. {
  797. "headers": [
  798. { ":method": "POST" },
  799. { ":path": "/" }
  800. ]
  801. }
  802. ]
  803. }
  804. With the ``-t`` option, the program can accept more familiar HTTP/1 style
  805. header field blocks. Each header set is delimited by an empty line:
  806. Example:
  807. .. code-block:: text
  808. :method: GET
  809. :scheme: https
  810. :path: /
  811. :method: POST
  812. user-agent: nghttp2
  813. The output is in JSON object. It should include a ``cases`` key and its
  814. value is an array of JSON objects, which has at least the following keys:
  815. seq
  816. The index of header set in the input.
  817. input_length
  818. The sum of the length of the name/value pairs in the input.
  819. output_length
  820. The length of the compressed header block.
  821. percentage_of_original_size
  822. ``output_length`` / ``input_length`` * 100
  823. wire
  824. The compressed header block as a hex string.
  825. headers
  826. The input header set.
  827. header_table_size
  828. The header table size adjusted before deflating the header set.
  829. Examples:
  830. .. code-block:: json
  831. {
  832. "cases":
  833. [
  834. {
  835. "seq": 0,
  836. "input_length": 66,
  837. "output_length": 20,
  838. "percentage_of_original_size": 30.303030303030305,
  839. "wire": "01881f3468e5891afcbf83868a3d856659c62e3f",
  840. "headers": [
  841. {
  842. ":authority": "example.org"
  843. },
  844. {
  845. ":method": "GET"
  846. },
  847. {
  848. ":path": "/"
  849. },
  850. {
  851. ":scheme": "https"
  852. },
  853. {
  854. "user-agent": "nghttp2"
  855. }
  856. ],
  857. "header_table_size": 4096
  858. }
  859. ,
  860. {
  861. "seq": 1,
  862. "input_length": 74,
  863. "output_length": 10,
  864. "percentage_of_original_size": 13.513513513513514,
  865. "wire": "88448504252dd5918485",
  866. "headers": [
  867. {
  868. ":authority": "example.org"
  869. },
  870. {
  871. ":method": "POST"
  872. },
  873. {
  874. ":path": "/account"
  875. },
  876. {
  877. ":scheme": "https"
  878. },
  879. {
  880. "user-agent": "nghttp2"
  881. }
  882. ],
  883. "header_table_size": 4096
  884. }
  885. ]
  886. }
  887. The output can be used as the input for ``inflatehd`` and
  888. ``deflatehd``.
  889. With the ``-d`` option, the extra ``header_table`` key is added and its
  890. associated value includes the state of dynamic header table after the
  891. corresponding header set was processed. The value includes at least
  892. the following keys:
  893. entries
  894. The entry in the header table. If ``referenced`` is ``true``, it
  895. is in the reference set. The ``size`` includes the overhead (32
  896. bytes). The ``index`` corresponds to the index of header table.
  897. The ``name`` is the header field name and the ``value`` is the
  898. header field value.
  899. size
  900. The sum of the spaces entries occupied, this includes the
  901. entry overhead.
  902. max_size
  903. The maximum header table size.
  904. deflate_size
  905. The sum of the spaces entries occupied within
  906. ``max_deflate_size``.
  907. max_deflate_size
  908. The maximum header table size the encoder uses. This can be smaller
  909. than ``max_size``. In this case, the encoder only uses up to first
  910. ``max_deflate_size`` buffer. Since the header table size is still
  911. ``max_size``, the encoder has to keep track of entries outside the
  912. ``max_deflate_size`` but inside the ``max_size`` and make sure
  913. that they are no longer referenced.
  914. Example:
  915. .. code-block:: json
  916. {
  917. "cases":
  918. [
  919. {
  920. "seq": 0,
  921. "input_length": 66,
  922. "output_length": 20,
  923. "percentage_of_original_size": 30.303030303030305,
  924. "wire": "01881f3468e5891afcbf83868a3d856659c62e3f",
  925. "headers": [
  926. {
  927. ":authority": "example.org"
  928. },
  929. {
  930. ":method": "GET"
  931. },
  932. {
  933. ":path": "/"
  934. },
  935. {
  936. ":scheme": "https"
  937. },
  938. {
  939. "user-agent": "nghttp2"
  940. }
  941. ],
  942. "header_table_size": 4096,
  943. "header_table": {
  944. "entries": [
  945. {
  946. "index": 1,
  947. "name": "user-agent",
  948. "value": "nghttp2",
  949. "referenced": true,
  950. "size": 49
  951. },
  952. {
  953. "index": 2,
  954. "name": ":scheme",
  955. "value": "https",
  956. "referenced": true,
  957. "size": 44
  958. },
  959. {
  960. "index": 3,
  961. "name": ":path",
  962. "value": "/",
  963. "referenced": true,
  964. "size": 38
  965. },
  966. {
  967. "index": 4,
  968. "name": ":method",
  969. "value": "GET",
  970. "referenced": true,
  971. "size": 42
  972. },
  973. {
  974. "index": 5,
  975. "name": ":authority",
  976. "value": "example.org",
  977. "referenced": true,
  978. "size": 53
  979. }
  980. ],
  981. "size": 226,
  982. "max_size": 4096,
  983. "deflate_size": 226,
  984. "max_deflate_size": 4096
  985. }
  986. }
  987. ,
  988. {
  989. "seq": 1,
  990. "input_length": 74,
  991. "output_length": 10,
  992. "percentage_of_original_size": 13.513513513513514,
  993. "wire": "88448504252dd5918485",
  994. "headers": [
  995. {
  996. ":authority": "example.org"
  997. },
  998. {
  999. ":method": "POST"
  1000. },
  1001. {
  1002. ":path": "/account"
  1003. },
  1004. {
  1005. ":scheme": "https"
  1006. },
  1007. {
  1008. "user-agent": "nghttp2"
  1009. }
  1010. ],
  1011. "header_table_size": 4096,
  1012. "header_table": {
  1013. "entries": [
  1014. {
  1015. "index": 1,
  1016. "name": ":method",
  1017. "value": "POST",
  1018. "referenced": true,
  1019. "size": 43
  1020. },
  1021. {
  1022. "index": 2,
  1023. "name": "user-agent",
  1024. "value": "nghttp2",
  1025. "referenced": true,
  1026. "size": 49
  1027. },
  1028. {
  1029. "index": 3,
  1030. "name": ":scheme",
  1031. "value": "https",
  1032. "referenced": true,
  1033. "size": 44
  1034. },
  1035. {
  1036. "index": 4,
  1037. "name": ":path",
  1038. "value": "/",
  1039. "referenced": false,
  1040. "size": 38
  1041. },
  1042. {
  1043. "index": 5,
  1044. "name": ":method",
  1045. "value": "GET",
  1046. "referenced": false,
  1047. "size": 42
  1048. },
  1049. {
  1050. "index": 6,
  1051. "name": ":authority",
  1052. "value": "example.org",
  1053. "referenced": true,
  1054. "size": 53
  1055. }
  1056. ],
  1057. "size": 269,
  1058. "max_size": 4096,
  1059. "deflate_size": 269,
  1060. "max_deflate_size": 4096
  1061. }
  1062. }
  1063. ]
  1064. }
  1065. inflatehd - header decompressor
  1066. +++++++++++++++++++++++++++++++
  1067. The ``inflatehd`` program reads JSON data from stdin and outputs decompressed
  1068. name/value pairs in JSON.
  1069. The root JSON object must include the ``cases`` key. Its value has to
  1070. include the sequence of compressed header blocks. They share the same
  1071. compression context and are processed in the order they appear. Each
  1072. item in the sequence is a JSON object and it must have at least a
  1073. ``wire`` key. Its value is a compressed header block as a hex string.
  1074. Example:
  1075. .. code-block:: json
  1076. {
  1077. "cases":
  1078. [
  1079. { "wire": "8285" },
  1080. { "wire": "8583" }
  1081. ]
  1082. }
  1083. The output is a JSON object. It should include a ``cases`` key and its
  1084. value is an array of JSON objects, which has at least following keys:
  1085. seq
  1086. The index of the header set in the input.
  1087. headers
  1088. A JSON array that includes decompressed name/value pairs.
  1089. wire
  1090. The compressed header block as a hex string.
  1091. header_table_size
  1092. The header table size adjusted before inflating compressed header
  1093. block.
  1094. Example:
  1095. .. code-block:: json
  1096. {
  1097. "cases":
  1098. [
  1099. {
  1100. "seq": 0,
  1101. "wire": "01881f3468e5891afcbf83868a3d856659c62e3f",
  1102. "headers": [
  1103. {
  1104. ":authority": "example.org"
  1105. },
  1106. {
  1107. ":method": "GET"
  1108. },
  1109. {
  1110. ":path": "/"
  1111. },
  1112. {
  1113. ":scheme": "https"
  1114. },
  1115. {
  1116. "user-agent": "nghttp2"
  1117. }
  1118. ],
  1119. "header_table_size": 4096
  1120. }
  1121. ,
  1122. {
  1123. "seq": 1,
  1124. "wire": "88448504252dd5918485",
  1125. "headers": [
  1126. {
  1127. ":method": "POST"
  1128. },
  1129. {
  1130. ":path": "/account"
  1131. },
  1132. {
  1133. "user-agent": "nghttp2"
  1134. },
  1135. {
  1136. ":scheme": "https"
  1137. },
  1138. {
  1139. ":authority": "example.org"
  1140. }
  1141. ],
  1142. "header_table_size": 4096
  1143. }
  1144. ]
  1145. }
  1146. The output can be used as the input for ``deflatehd`` and
  1147. ``inflatehd``.
  1148. With the ``-d`` option, the extra ``header_table`` key is added and its
  1149. associated value includes the state of the dynamic header table after the
  1150. corresponding header set was processed. The format is the same as
  1151. ``deflatehd``.
  1152. Contribution
  1153. ------------
  1154. [This text was composed based on 1.2. License section of curl/libcurl
  1155. project.]
  1156. When contributing with code, you agree to put your changes and new
  1157. code under the same license nghttp2 is already using unless stated and
  1158. agreed otherwise.
  1159. When changing existing source code, do not alter the copyright of
  1160. the original file(s). The copyright will still be owned by the
  1161. original creator(s) or those who have been assigned copyright by the
  1162. original author(s).
  1163. By submitting a patch to the nghttp2 project, you (or your employer, as
  1164. the case may be) agree to assign the copyright of your submission to us.
  1165. .. the above really needs to be reworded to pass legal muster.
  1166. We will credit you for your
  1167. changes as far as possible, to give credit but also to keep a trace
  1168. back to who made what changes. Please always provide us with your
  1169. full real name when contributing!
  1170. See `Contribution Guidelines
  1171. <https://nghttp2.org/documentation/contribute.html>`_ for more
  1172. details.
  1173. Versioning
  1174. ----------
  1175. In general, we follow `Semantic Versioning <http://semver.org/>`_.
  1176. We may release PATCH releases between the regular releases, mainly for
  1177. severe security bug fixes.
  1178. We have no plan to break API compatibility changes involving soname
  1179. bump, so MAJOR version will stay 1 for the foreseeable future.
  1180. License
  1181. -------
  1182. The MIT License