README.rst 55 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807808809810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841842843844845846847848849850851852853854855856857858859860861862863864865866867868869870871872873874875876877878879880881882883884885886887888889890891892893894895896897898899900901902903904905906907908909910911912913914915916917918919920921922923924925926927928929930931932933934935936937938939940941942943944945946947948949950951952953954955956957958959960961962963964965966967968969970971972973974975976977978979980981982983984985986987988989990991992993994995996997998999100010011002100310041005100610071008100910101011101210131014101510161017101810191020102110221023102410251026102710281029103010311032103310341035103610371038103910401041104210431044104510461047104810491050105110521053105410551056105710581059106010611062106310641065106610671068106910701071107210731074107510761077107810791080108110821083108410851086108710881089109010911092109310941095109610971098109911001101110211031104110511061107110811091110111111121113111411151116111711181119112011211122112311241125112611271128112911301131113211331134113511361137113811391140114111421143114411451146114711481149115011511152115311541155115611571158115911601161116211631164116511661167116811691170117111721173117411751176117711781179118011811182118311841185118611871188118911901191119211931194119511961197119811991200120112021203120412051206120712081209121012111212121312141215121612171218121912201221122212231224122512261227122812291230123112321233123412351236123712381239124012411242124312441245124612471248124912501251125212531254125512561257125812591260126112621263126412651266126712681269127012711272127312741275127612771278127912801281128212831284128512861287128812891290129112921293129412951296129712981299130013011302130313041305130613071308130913101311131213131314131513161317131813191320132113221323132413251326132713281329133013311332133313341335133613371338133913401341134213431344134513461347134813491350135113521353135413551356135713581359136013611362136313641365136613671368136913701371137213731374137513761377137813791380138113821383138413851386138713881389139013911392139313941395139613971398139914001401140214031404140514061407140814091410141114121413141414151416141714181419142014211422142314241425142614271428142914301431143214331434143514361437143814391440144114421443144414451446144714481449145014511452145314541455145614571458145914601461146214631464146514661467146814691470147114721473147414751476147714781479148014811482148314841485148614871488148914901491149214931494149514961497149814991500150115021503150415051506150715081509151015111512151315141515151615171518151915201521152215231524152515261527152815291530153115321533153415351536153715381539154015411542154315441545154615471548154915501551155215531554155515561557155815591560156115621563156415651566156715681569157015711572157315741575157615771578157915801581158215831584158515861587158815891590159115921593159415951596159715981599160016011602160316041605160616071608160916101611161216131614161516161617161816191620162116221623162416251626162716281629163016311632163316341635163616371638163916401641164216431644164516461647164816491650165116521653165416551656165716581659166016611662166316641665166616671668166916701671167216731674167516761677167816791680168116821683168416851686168716881689169016911692169316941695169616971698169917001701
  1. nghttp2 - HTTP/2 C Library
  2. ==========================
  3. This is an implementation of the Hypertext Transfer Protocol version 2
  4. in C.
  5. The framing layer of HTTP/2 is implemented as a reusable C library.
  6. On top of that, we have implemented an HTTP/2 client, server and
  7. proxy. We have also developed load test and benchmarking tools for
  8. HTTP/2.
  9. An HPACK encoder and decoder are available as a public API.
  10. An experimental high level C++ library is also available.
  11. We have Python bindings of this library, but we do not have full
  12. code coverage yet.
  13. Development Status
  14. ------------------
  15. We have implemented `RFC 7540 <https://tools.ietf.org/html/rfc7540>`_
  16. HTTP/2 and `RFC 7541 <https://tools.ietf.org/html/rfc7541>`_ HPACK -
  17. Header Compression for HTTP/2
  18. The nghttp2 code base was forked from the spdylay
  19. (https://github.com/tatsuhiro-t/spdylay) project.
  20. Public Test Server
  21. ------------------
  22. The following endpoints are available to try out our nghttp2
  23. implementation.
  24. * https://nghttp2.org/ (TLS + ALPN/NPN and HTTP/3)
  25. This endpoint supports ``h2``, ``h2-16``, ``h2-14``, and
  26. ``http/1.1`` via ALPN/NPN and requires TLSv1.2 for HTTP/2
  27. connection.
  28. It also supports HTTP/3.
  29. * http://nghttp2.org/ (HTTP Upgrade and HTTP/2 Direct)
  30. ``h2c`` and ``http/1.1``.
  31. Requirements
  32. ------------
  33. The following package is required to build the libnghttp2 library:
  34. * pkg-config >= 0.20
  35. To build and run the unit test programs, the following package is
  36. required:
  37. * cunit >= 2.1
  38. To build the documentation, you need to install:
  39. * sphinx (http://sphinx-doc.org/)
  40. If you need libnghttp2 (C library) only, then the above packages are
  41. all you need. Use ``--enable-lib-only`` to ensure that only
  42. libnghttp2 is built. This avoids potential build error related to
  43. building bundled applications.
  44. To build and run the application programs (``nghttp``, ``nghttpd``,
  45. ``nghttpx`` and ``h2load``) in the ``src`` directory, the following packages
  46. are required:
  47. * OpenSSL >= 1.0.1
  48. * libev >= 4.11
  49. * zlib >= 1.2.3
  50. * libc-ares >= 1.7.5
  51. ALPN support requires OpenSSL >= 1.0.2 (released 22 January 2015).
  52. LibreSSL >= 2.2.0 can be used instead of OpenSSL, but OpenSSL has more
  53. features than LibreSSL at the time of this writing.
  54. To enable ``-a`` option (getting linked assets from the downloaded
  55. resource) in ``nghttp``, the following package is required:
  56. * libxml2 >= 2.6.26
  57. To enable systemd support in nghttpx, the following package is
  58. required:
  59. * libsystemd-dev >= 209
  60. The HPACK tools require the following package:
  61. * jansson >= 2.5
  62. To build sources under the examples directory, libevent is required:
  63. * libevent-openssl >= 2.0.8
  64. To mitigate heap fragmentation in long running server programs
  65. (``nghttpd`` and ``nghttpx``), jemalloc is recommended:
  66. * jemalloc
  67. .. note::
  68. Alpine Linux currently does not support malloc replacement
  69. due to musl limitations. See details in issue `#762 <https://github.com/nghttp2/nghttp2/issues/762>`_.
  70. libnghttp2_asio C++ library requires the following packages:
  71. * libboost-dev >= 1.54.0
  72. * libboost-thread-dev >= 1.54.0
  73. The Python bindings require the following packages:
  74. * cython >= 0.19
  75. * python >= 3.8
  76. * python-setuptools
  77. If you are using Ubuntu 16.04 LTS (Xenial Xerus) or Debian 8 (jessie)
  78. and above, run the following to install the required packages:
  79. .. code-block:: text
  80. sudo apt-get install g++ make binutils autoconf automake autotools-dev libtool pkg-config \
  81. zlib1g-dev libcunit1-dev libssl-dev libxml2-dev libev-dev libevent-dev libjansson-dev \
  82. libc-ares-dev libjemalloc-dev libsystemd-dev \
  83. cython python3-dev python-setuptools
  84. To enable mruby support for nghttpx, `mruby
  85. <https://github.com/mruby/mruby>`_ is required. We need to build
  86. mruby with C++ ABI explicitly turned on, and probably need other
  87. mrgems, mruby is manged by git submodule under third-party/mruby
  88. directory. Currently, mruby support for nghttpx is disabled by
  89. default. To enable mruby support, use ``--with-mruby`` configure
  90. option. Note that at the time of this writing, libmruby-dev and mruby
  91. packages in Debian/Ubuntu are not usable for nghttp2, since they do
  92. not enable C++ ABI. To build mruby, the following packages are
  93. required:
  94. * ruby
  95. * bison
  96. nghttpx supports `neverbleed <https://github.com/h2o/neverbleed>`_,
  97. privilege separation engine for OpenSSL / LibreSSL. In short, it
  98. minimizes the risk of private key leakage when serious bug like
  99. Heartbleed is exploited. The neverbleed is disabled by default. To
  100. enable it, use ``--with-neverbleed`` configure option.
  101. To enable the experimental HTTP/3 support for h2load and nghttpx, the
  102. following libraries are required:
  103. * `OpenSSL with QUIC support
  104. <https://github.com/quictls/openssl/tree/OpenSSL_1_1_1m+quic>`_; or
  105. `BoringSSL <https://boringssl.googlesource.com/boringssl/>`_ (commit
  106. 36a41bf0bf2dd3176f8780e09c03585351f29963)
  107. * `ngtcp2 <https://github.com/ngtcp2/ngtcp2>`_ >= 0.2.0
  108. * `nghttp3 <https://github.com/ngtcp2/nghttp3>`_ >= 0.2.0
  109. Use ``--enable-http3`` configure option to enable HTTP/3 feature for
  110. h2load and nghttpx.
  111. In order to build optional eBPF program to direct an incoming QUIC UDP
  112. datagram to a correct socket for nghttpx, the following libraries are
  113. required:
  114. * libbpf-dev >= 0.7.0
  115. Use ``--with-libbpf`` configure option to build eBPF program.
  116. libelf-dev is needed to build libbpf.
  117. For Ubuntu 20.04, you can build libbpf from `the source code
  118. <https://github.com/libbpf/libbpf/releases/tag/v0.7.0>`_. nghttpx
  119. requires eBPF program for reloading its configuration and hot swapping
  120. its executable.
  121. Compiling libnghttp2 C source code requires a C99 compiler. gcc 4.8
  122. is known to be adequate. In order to compile the C++ source code, gcc
  123. >= 6.0 or clang >= 6.0 is required. C++ source code requires C++14
  124. language features.
  125. .. note::
  126. To enable mruby support in nghttpx, and use ``--with-mruby``
  127. configure option.
  128. .. note::
  129. Mac OS X users may need the ``--disable-threads`` configure option to
  130. disable multi-threading in nghttpd, nghttpx and h2load to prevent
  131. them from crashing. A patch is welcome to make multi threading work
  132. on Mac OS X platform.
  133. .. note::
  134. To compile the associated applications (nghttp, nghttpd, nghttpx
  135. and h2load), you must use the ``--enable-app`` configure option and
  136. ensure that the specified requirements above are met. Normally,
  137. configure script checks required dependencies to build these
  138. applications, and enable ``--enable-app`` automatically, so you
  139. don't have to use it explicitly. But if you found that
  140. applications were not built, then using ``--enable-app`` may find
  141. that cause, such as the missing dependency.
  142. .. note::
  143. In order to detect third party libraries, pkg-config is used
  144. (however we don't use pkg-config for some libraries (e.g., libev)).
  145. By default, pkg-config searches ``*.pc`` file in the standard
  146. locations (e.g., /usr/lib/pkgconfig). If it is necessary to use
  147. ``*.pc`` file in the custom location, specify paths to
  148. ``PKG_CONFIG_PATH`` environment variable, and pass it to configure
  149. script, like so:
  150. .. code-block:: text
  151. $ ./configure PKG_CONFIG_PATH=/path/to/pkgconfig
  152. For pkg-config managed libraries, ``*_CFLAG`` and ``*_LIBS``
  153. environment variables are defined (e.g., ``OPENSSL_CFLAGS``,
  154. ``OPENSSL_LIBS``). Specifying non-empty string to these variables
  155. completely overrides pkg-config. In other words, if they are
  156. specified, pkg-config is not used for detection, and user is
  157. responsible to specify the correct values to these variables. For
  158. complete list of these variables, run ``./configure -h``.
  159. Building nghttp2 from release tar archive
  160. -----------------------------------------
  161. The nghttp2 project regularly releases tar archives which includes
  162. nghttp2 source code, and generated build files. They can be
  163. downloaded from `Releases
  164. <https://github.com/nghttp2/nghttp2/releases>`_ page.
  165. Building nghttp2 from git requires autotools development packages.
  166. Building from tar archives does not require them, and thus it is much
  167. easier. The usual build step is as follows:
  168. .. code-block:: text
  169. $ tar xf nghttp2-X.Y.Z.tar.bz2
  170. $ cd nghttp2-X.Y.Z
  171. $ ./configure
  172. $ make
  173. Building from git
  174. -----------------
  175. Building from git is easy, but please be sure that at least autoconf 2.68 is
  176. used:
  177. .. code-block:: text
  178. $ git submodule update --init
  179. $ autoreconf -i
  180. $ automake
  181. $ autoconf
  182. $ ./configure
  183. $ make
  184. Notes for building on Windows (MSVC)
  185. ------------------------------------
  186. The easiest way to build native Windows nghttp2 dll is use `cmake
  187. <https://cmake.org/>`_. The free version of `Visual C++ Build Tools
  188. <http://landinghub.visualstudio.com/visual-cpp-build-tools>`_ works
  189. fine.
  190. 1. Install cmake for windows
  191. 2. Open "Visual C++ ... Native Build Tool Command Prompt", and inside
  192. nghttp2 directly, run ``cmake``.
  193. 3. Then run ``cmake --build`` to build library.
  194. 4. nghttp2.dll, nghttp2.lib, nghttp2.exp are placed under lib directory.
  195. Note that the above steps most likely produce nghttp2 library only.
  196. No bundled applications are compiled.
  197. Notes for building on Windows (Mingw/Cygwin)
  198. --------------------------------------------
  199. Under Mingw environment, you can only compile the library, it's
  200. ``libnghttp2-X.dll`` and ``libnghttp2.a``.
  201. If you want to compile the applications(``h2load``, ``nghttp``,
  202. ``nghttpx``, ``nghttpd``), you need to use the Cygwin environment.
  203. Under Cygwin environment, to compile the applications you need to
  204. compile and install the libev first.
  205. Secondly, you need to undefine the macro ``__STRICT_ANSI__``, if you
  206. not, the functions ``fdopen``, ``fileno`` and ``strptime`` will not
  207. available.
  208. the sample command like this:
  209. .. code-block:: text
  210. $ export CFLAGS="-U__STRICT_ANSI__ -I$libev_PREFIX/include -L$libev_PREFIX/lib"
  211. $ export CXXFLAGS=$CFLAGS
  212. $ ./configure
  213. $ make
  214. If you want to compile the applications under ``examples/``, you need
  215. to remove or rename the ``event.h`` from libev's installation, because
  216. it conflicts with libevent's installation.
  217. Notes for installation on Linux systems
  218. --------------------------------------------
  219. After installing nghttp2 tool suite with ``make install`` one might experience a similar error:
  220. .. code-block:: text
  221. nghttpx: error while loading shared libraries: libnghttp2.so.14: cannot open shared object file: No such file or directory
  222. This means that the tool is unable to locate the ``libnghttp2.so`` shared library.
  223. To update the shared library cache run ``sudo ldconfig``.
  224. Building the documentation
  225. --------------------------
  226. .. note::
  227. Documentation is still incomplete.
  228. To build the documentation, run:
  229. .. code-block:: text
  230. $ make html
  231. The documents will be generated under ``doc/manual/html/``.
  232. The generated documents will not be installed with ``make install``.
  233. The online documentation is available at
  234. https://nghttp2.org/documentation/
  235. Build HTTP/3 enabled h2load and nghttpx
  236. ---------------------------------------
  237. To build h2load and nghttpx with HTTP/3 feature enabled, run the
  238. configure script with ``--enable-http3``.
  239. For nghttpx to reload configurations and swapping its executable while
  240. gracefully terminating old worker processes, eBPF is required. Run
  241. the configure script with ``--enable-http3 --with-libbpf`` to build
  242. eBPF program. The QUIC keying material must be set with
  243. ``--frontend-quic-secret-file`` in order to keep the existing
  244. connections alive during reload.
  245. The detailed steps to build HTTP/3 enabled h2load and nghttpx follow.
  246. Build custom OpenSSL:
  247. .. code-block:: text
  248. $ git clone --depth 1 -b OpenSSL_1_1_1m+quic https://github.com/quictls/openssl
  249. $ cd openssl
  250. $ ./config --prefix=$PWD/build --openssldir=/etc/ssl
  251. $ make -j$(nproc)
  252. $ make install_sw
  253. $ cd ..
  254. Build nghttp3:
  255. .. code-block:: text
  256. $ git clone --depth 1 -b v0.2.0 https://github.com/ngtcp2/nghttp3
  257. $ cd nghttp3
  258. $ autoreconf -i
  259. $ ./configure --prefix=$PWD/build --enable-lib-only
  260. $ make -j$(nproc)
  261. $ make install
  262. $ cd ..
  263. Build ngtcp2:
  264. .. code-block:: text
  265. $ git clone --depth 1 -b v0.2.0 https://github.com/ngtcp2/ngtcp2
  266. $ cd ngtcp2
  267. $ autoreconf -i
  268. $ ./configure --prefix=$PWD/build --enable-lib-only \
  269. PKG_CONFIG_PATH="$PWD/../openssl/build/lib/pkgconfig"
  270. $ make -j$(nproc)
  271. $ make install
  272. $ cd ..
  273. If your Linux distribution does not have libbpf-dev >= 0.7.0, build
  274. from source:
  275. .. code-block:: text
  276. $ git clone --depth 1 -b v0.7.0 https://github.com/libbpf/libbpf
  277. $ cd libbpf
  278. $ PREFIX=$PWD/build make -C src install
  279. $ cd ..
  280. Build nghttp2:
  281. .. code-block:: text
  282. $ git clone https://github.com/nghttp2/nghttp2
  283. $ cd nghttp2
  284. $ git submodule update --init
  285. $ autoreconf -i
  286. $ ./configure --with-mruby --with-neverbleed --enable-http3 --with-libbpf \
  287. --disable-python-bindings \
  288. CC=clang-12 CXX=clang++-12 \
  289. PKG_CONFIG_PATH="$PWD/../openssl/build/lib/pkgconfig:$PWD/../nghttp3/build/lib/pkgconfig:$PWD/../ngtcp2/build/lib/pkgconfig:$PWD/../libbpf/build/lib64/pkgconfig" \
  290. LDFLAGS="$LDFLAGS -Wl,-rpath,$PWD/../openssl/build/lib -Wl,-rpath,$PWD/../libbpf/build/lib64"
  291. $ make -j$(nproc)
  292. The eBPF program ``reuseport_kern.o`` should be found under bpf
  293. directory. Pass ``--quic-bpf-program-file=bpf/reuseport_kern.o``
  294. option to nghttpx to load it. See also `HTTP/3 section in nghttpx -
  295. HTTP/2 proxy - HOW-TO
  296. <https://nghttp2.org/documentation/nghttpx-howto.html#http-3>`_.
  297. Unit tests
  298. ----------
  299. Unit tests are done by simply running ``make check``.
  300. Integration tests
  301. -----------------
  302. We have the integration tests for the nghttpx proxy server. The tests are
  303. written in the `Go programming language <http://golang.org/>`_ and uses
  304. its testing framework. We depend on the following libraries:
  305. * golang.org/x/net/http2
  306. * golang.org/x/net/websocket
  307. * https://github.com/tatsuhiro-t/go-nghttp2
  308. Go modules will download these dependencies automatically.
  309. To run the tests, run the following command under
  310. ``integration-tests`` directory:
  311. .. code-block:: text
  312. $ make it
  313. Inside the tests, we use port 3009 to run the test subject server.
  314. Migration from v0.7.15 or earlier
  315. ---------------------------------
  316. nghttp2 v1.0.0 introduced several backward incompatible changes. In
  317. this section, we describe these changes and how to migrate to v1.0.0.
  318. ALPN protocol ID is now ``h2`` and ``h2c``
  319. ++++++++++++++++++++++++++++++++++++++++++
  320. Previously we announced ``h2-14`` and ``h2c-14``. v1.0.0 implements
  321. final protocol version, and we changed ALPN ID to ``h2`` and ``h2c``.
  322. The macros ``NGHTTP2_PROTO_VERSION_ID``,
  323. ``NGHTTP2_PROTO_VERSION_ID_LEN``,
  324. ``NGHTTP2_CLEARTEXT_PROTO_VERSION_ID``, and
  325. ``NGHTTP2_CLEARTEXT_PROTO_VERSION_ID_LEN`` have been updated to
  326. reflect this change.
  327. Basically, existing applications do not have to do anything, just
  328. recompiling is enough for this change.
  329. Use word "client magic" where we use "client connection preface"
  330. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  331. We use "client connection preface" to mean first 24 bytes of client
  332. connection preface. This is technically not correct, since client
  333. connection preface is composed of 24 bytes client magic byte string
  334. followed by SETTINGS frame. For clarification, we call "client magic"
  335. for this 24 bytes byte string and updated API.
  336. * ``NGHTTP2_CLIENT_CONNECTION_PREFACE`` was replaced with
  337. ``NGHTTP2_CLIENT_MAGIC``.
  338. * ``NGHTTP2_CLIENT_CONNECTION_PREFACE_LEN`` was replaced with
  339. ``NGHTTP2_CLIENT_MAGIC_LEN``.
  340. * ``NGHTTP2_BAD_PREFACE`` was renamed as ``NGHTTP2_BAD_CLIENT_MAGIC``
  341. The already deprecated ``NGHTTP2_CLIENT_CONNECTION_HEADER`` and
  342. ``NGHTTP2_CLIENT_CONNECTION_HEADER_LEN`` were removed.
  343. If application uses these macros, just replace old ones with new ones.
  344. Since v1.0.0, client magic is sent by library (see next subsection),
  345. so client application may just remove these macro use.
  346. Client magic is sent by library
  347. +++++++++++++++++++++++++++++++
  348. Previously nghttp2 library did not send client magic, which is first
  349. 24 bytes byte string of client connection preface, and client
  350. applications have to send it by themselves. Since v1.0.0, client
  351. magic is sent by library via first call of ``nghttp2_session_send()``
  352. or ``nghttp2_session_mem_send()``.
  353. The client applications which send client magic must remove the
  354. relevant code.
  355. Remove HTTP Alternative Services (Alt-Svc) related code
  356. +++++++++++++++++++++++++++++++++++++++++++++++++++++++
  357. Alt-Svc specification is not finalized yet. To make our API stable,
  358. we have decided to remove all Alt-Svc related API from nghttp2.
  359. * ``NGHTTP2_EXT_ALTSVC`` was removed.
  360. * ``nghttp2_ext_altsvc`` was removed.
  361. We have already removed the functionality of Alt-Svc in v0.7 series
  362. and they have been essentially noop. The application using these
  363. macro and struct, remove those lines.
  364. Use nghttp2_error in nghttp2_on_invalid_frame_recv_callback
  365. +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  366. Previously ``nghttp2_on_invalid_frame_recv_cb_called`` took the
  367. ``error_code``, defined in ``nghttp2_error_code``, as parameter. But
  368. they are not detailed enough to debug. Therefore, we decided to use
  369. more detailed ``nghttp2_error`` values instead.
  370. The application using this callback should update the callback
  371. signature. If it treats ``error_code`` as HTTP/2 error code, update
  372. the code so that it is treated as ``nghttp2_error``.
  373. Receive client magic by default
  374. +++++++++++++++++++++++++++++++
  375. Previously nghttp2 did not process client magic (24 bytes byte
  376. string). To make it deal with it, we had to use
  377. ``nghttp2_option_set_recv_client_preface()``. Since v1.0.0, nghttp2
  378. processes client magic by default and
  379. ``nghttp2_option_set_recv_client_preface()`` was removed.
  380. Some application may want to disable this behaviour, so we added
  381. ``nghttp2_option_set_no_recv_client_magic()`` to achieve this.
  382. The application using ``nghttp2_option_set_recv_client_preface()``
  383. with nonzero value, just remove it.
  384. The application using ``nghttp2_option_set_recv_client_preface()``
  385. with zero value or not using it must use
  386. ``nghttp2_option_set_no_recv_client_magic()`` with nonzero value.
  387. Client, Server and Proxy programs
  388. ---------------------------------
  389. The ``src`` directory contains the HTTP/2 client, server and proxy programs.
  390. nghttp - client
  391. +++++++++++++++
  392. ``nghttp`` is a HTTP/2 client. It can connect to the HTTP/2 server
  393. with prior knowledge, HTTP Upgrade and NPN/ALPN TLS extension.
  394. It has verbose output mode for framing information. Here is sample
  395. output from ``nghttp`` client:
  396. .. code-block:: text
  397. $ nghttp -nv https://nghttp2.org
  398. [ 0.190] Connected
  399. The negotiated protocol: h2
  400. [ 0.212] recv SETTINGS frame <length=12, flags=0x00, stream_id=0>
  401. (niv=2)
  402. [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
  403. [SETTINGS_INITIAL_WINDOW_SIZE(0x04):65535]
  404. [ 0.212] send SETTINGS frame <length=12, flags=0x00, stream_id=0>
  405. (niv=2)
  406. [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
  407. [SETTINGS_INITIAL_WINDOW_SIZE(0x04):65535]
  408. [ 0.212] send SETTINGS frame <length=0, flags=0x01, stream_id=0>
  409. ; ACK
  410. (niv=0)
  411. [ 0.212] send PRIORITY frame <length=5, flags=0x00, stream_id=3>
  412. (dep_stream_id=0, weight=201, exclusive=0)
  413. [ 0.212] send PRIORITY frame <length=5, flags=0x00, stream_id=5>
  414. (dep_stream_id=0, weight=101, exclusive=0)
  415. [ 0.212] send PRIORITY frame <length=5, flags=0x00, stream_id=7>
  416. (dep_stream_id=0, weight=1, exclusive=0)
  417. [ 0.212] send PRIORITY frame <length=5, flags=0x00, stream_id=9>
  418. (dep_stream_id=7, weight=1, exclusive=0)
  419. [ 0.212] send PRIORITY frame <length=5, flags=0x00, stream_id=11>
  420. (dep_stream_id=3, weight=1, exclusive=0)
  421. [ 0.212] send HEADERS frame <length=39, flags=0x25, stream_id=13>
  422. ; END_STREAM | END_HEADERS | PRIORITY
  423. (padlen=0, dep_stream_id=11, weight=16, exclusive=0)
  424. ; Open new stream
  425. :method: GET
  426. :path: /
  427. :scheme: https
  428. :authority: nghttp2.org
  429. accept: */*
  430. accept-encoding: gzip, deflate
  431. user-agent: nghttp2/1.0.1-DEV
  432. [ 0.221] recv SETTINGS frame <length=0, flags=0x01, stream_id=0>
  433. ; ACK
  434. (niv=0)
  435. [ 0.221] recv (stream_id=13) :method: GET
  436. [ 0.221] recv (stream_id=13) :scheme: https
  437. [ 0.221] recv (stream_id=13) :path: /stylesheets/screen.css
  438. [ 0.221] recv (stream_id=13) :authority: nghttp2.org
  439. [ 0.221] recv (stream_id=13) accept-encoding: gzip, deflate
  440. [ 0.222] recv (stream_id=13) user-agent: nghttp2/1.0.1-DEV
  441. [ 0.222] recv PUSH_PROMISE frame <length=50, flags=0x04, stream_id=13>
  442. ; END_HEADERS
  443. (padlen=0, promised_stream_id=2)
  444. [ 0.222] recv (stream_id=13) :status: 200
  445. [ 0.222] recv (stream_id=13) date: Thu, 21 May 2015 16:38:14 GMT
  446. [ 0.222] recv (stream_id=13) content-type: text/html
  447. [ 0.222] recv (stream_id=13) last-modified: Fri, 15 May 2015 15:38:06 GMT
  448. [ 0.222] recv (stream_id=13) etag: W/"555612de-19f6"
  449. [ 0.222] recv (stream_id=13) link: </stylesheets/screen.css>; rel=preload; as=stylesheet
  450. [ 0.222] recv (stream_id=13) content-encoding: gzip
  451. [ 0.222] recv (stream_id=13) server: nghttpx nghttp2/1.0.1-DEV
  452. [ 0.222] recv (stream_id=13) via: 1.1 nghttpx
  453. [ 0.222] recv (stream_id=13) strict-transport-security: max-age=31536000
  454. [ 0.222] recv HEADERS frame <length=166, flags=0x04, stream_id=13>
  455. ; END_HEADERS
  456. (padlen=0)
  457. ; First response header
  458. [ 0.222] recv DATA frame <length=2601, flags=0x01, stream_id=13>
  459. ; END_STREAM
  460. [ 0.222] recv (stream_id=2) :status: 200
  461. [ 0.222] recv (stream_id=2) date: Thu, 21 May 2015 16:38:14 GMT
  462. [ 0.222] recv (stream_id=2) content-type: text/css
  463. [ 0.222] recv (stream_id=2) last-modified: Fri, 15 May 2015 15:38:06 GMT
  464. [ 0.222] recv (stream_id=2) etag: W/"555612de-9845"
  465. [ 0.222] recv (stream_id=2) content-encoding: gzip
  466. [ 0.222] recv (stream_id=2) server: nghttpx nghttp2/1.0.1-DEV
  467. [ 0.222] recv (stream_id=2) via: 1.1 nghttpx
  468. [ 0.222] recv (stream_id=2) strict-transport-security: max-age=31536000
  469. [ 0.222] recv HEADERS frame <length=32, flags=0x04, stream_id=2>
  470. ; END_HEADERS
  471. (padlen=0)
  472. ; First push response header
  473. [ 0.228] recv DATA frame <length=8715, flags=0x01, stream_id=2>
  474. ; END_STREAM
  475. [ 0.228] send GOAWAY frame <length=8, flags=0x00, stream_id=0>
  476. (last_stream_id=2, error_code=NO_ERROR(0x00), opaque_data(0)=[])
  477. The HTTP Upgrade is performed like so:
  478. .. code-block:: text
  479. $ nghttp -nvu http://nghttp2.org
  480. [ 0.011] Connected
  481. [ 0.011] HTTP Upgrade request
  482. GET / HTTP/1.1
  483. Host: nghttp2.org
  484. Connection: Upgrade, HTTP2-Settings
  485. Upgrade: h2c
  486. HTTP2-Settings: AAMAAABkAAQAAP__
  487. Accept: */*
  488. User-Agent: nghttp2/1.0.1-DEV
  489. [ 0.018] HTTP Upgrade response
  490. HTTP/1.1 101 Switching Protocols
  491. Connection: Upgrade
  492. Upgrade: h2c
  493. [ 0.018] HTTP Upgrade success
  494. [ 0.018] recv SETTINGS frame <length=12, flags=0x00, stream_id=0>
  495. (niv=2)
  496. [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
  497. [SETTINGS_INITIAL_WINDOW_SIZE(0x04):65535]
  498. [ 0.018] send SETTINGS frame <length=12, flags=0x00, stream_id=0>
  499. (niv=2)
  500. [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
  501. [SETTINGS_INITIAL_WINDOW_SIZE(0x04):65535]
  502. [ 0.018] send SETTINGS frame <length=0, flags=0x01, stream_id=0>
  503. ; ACK
  504. (niv=0)
  505. [ 0.018] send PRIORITY frame <length=5, flags=0x00, stream_id=3>
  506. (dep_stream_id=0, weight=201, exclusive=0)
  507. [ 0.018] send PRIORITY frame <length=5, flags=0x00, stream_id=5>
  508. (dep_stream_id=0, weight=101, exclusive=0)
  509. [ 0.018] send PRIORITY frame <length=5, flags=0x00, stream_id=7>
  510. (dep_stream_id=0, weight=1, exclusive=0)
  511. [ 0.018] send PRIORITY frame <length=5, flags=0x00, stream_id=9>
  512. (dep_stream_id=7, weight=1, exclusive=0)
  513. [ 0.018] send PRIORITY frame <length=5, flags=0x00, stream_id=11>
  514. (dep_stream_id=3, weight=1, exclusive=0)
  515. [ 0.018] send PRIORITY frame <length=5, flags=0x00, stream_id=1>
  516. (dep_stream_id=11, weight=16, exclusive=0)
  517. [ 0.019] recv (stream_id=1) :method: GET
  518. [ 0.019] recv (stream_id=1) :scheme: http
  519. [ 0.019] recv (stream_id=1) :path: /stylesheets/screen.css
  520. [ 0.019] recv (stream_id=1) host: nghttp2.org
  521. [ 0.019] recv (stream_id=1) user-agent: nghttp2/1.0.1-DEV
  522. [ 0.019] recv PUSH_PROMISE frame <length=49, flags=0x04, stream_id=1>
  523. ; END_HEADERS
  524. (padlen=0, promised_stream_id=2)
  525. [ 0.019] recv (stream_id=1) :status: 200
  526. [ 0.019] recv (stream_id=1) date: Thu, 21 May 2015 16:39:16 GMT
  527. [ 0.019] recv (stream_id=1) content-type: text/html
  528. [ 0.019] recv (stream_id=1) content-length: 6646
  529. [ 0.019] recv (stream_id=1) last-modified: Fri, 15 May 2015 15:38:06 GMT
  530. [ 0.019] recv (stream_id=1) etag: "555612de-19f6"
  531. [ 0.019] recv (stream_id=1) link: </stylesheets/screen.css>; rel=preload; as=stylesheet
  532. [ 0.019] recv (stream_id=1) accept-ranges: bytes
  533. [ 0.019] recv (stream_id=1) server: nghttpx nghttp2/1.0.1-DEV
  534. [ 0.019] recv (stream_id=1) via: 1.1 nghttpx
  535. [ 0.019] recv HEADERS frame <length=157, flags=0x04, stream_id=1>
  536. ; END_HEADERS
  537. (padlen=0)
  538. ; First response header
  539. [ 0.019] recv DATA frame <length=6646, flags=0x01, stream_id=1>
  540. ; END_STREAM
  541. [ 0.019] recv (stream_id=2) :status: 200
  542. [ 0.019] recv (stream_id=2) date: Thu, 21 May 2015 16:39:16 GMT
  543. [ 0.019] recv (stream_id=2) content-type: text/css
  544. [ 0.019] recv (stream_id=2) content-length: 38981
  545. [ 0.019] recv (stream_id=2) last-modified: Fri, 15 May 2015 15:38:06 GMT
  546. [ 0.019] recv (stream_id=2) etag: "555612de-9845"
  547. [ 0.019] recv (stream_id=2) accept-ranges: bytes
  548. [ 0.019] recv (stream_id=2) server: nghttpx nghttp2/1.0.1-DEV
  549. [ 0.019] recv (stream_id=2) via: 1.1 nghttpx
  550. [ 0.019] recv HEADERS frame <length=36, flags=0x04, stream_id=2>
  551. ; END_HEADERS
  552. (padlen=0)
  553. ; First push response header
  554. [ 0.026] recv DATA frame <length=16384, flags=0x00, stream_id=2>
  555. [ 0.027] recv DATA frame <length=7952, flags=0x00, stream_id=2>
  556. [ 0.027] send WINDOW_UPDATE frame <length=4, flags=0x00, stream_id=0>
  557. (window_size_increment=33343)
  558. [ 0.032] send WINDOW_UPDATE frame <length=4, flags=0x00, stream_id=2>
  559. (window_size_increment=33707)
  560. [ 0.032] recv DATA frame <length=14645, flags=0x01, stream_id=2>
  561. ; END_STREAM
  562. [ 0.032] recv SETTINGS frame <length=0, flags=0x01, stream_id=0>
  563. ; ACK
  564. (niv=0)
  565. [ 0.032] send GOAWAY frame <length=8, flags=0x00, stream_id=0>
  566. (last_stream_id=2, error_code=NO_ERROR(0x00), opaque_data(0)=[])
  567. Using the ``-s`` option, ``nghttp`` prints out some timing information for
  568. requests, sorted by completion time:
  569. .. code-block:: text
  570. $ nghttp -nas https://nghttp2.org/
  571. ***** Statistics *****
  572. Request timing:
  573. responseEnd: the time when last byte of response was received
  574. relative to connectEnd
  575. requestStart: the time just before first byte of request was sent
  576. relative to connectEnd. If '*' is shown, this was
  577. pushed by server.
  578. process: responseEnd - requestStart
  579. code: HTTP status code
  580. size: number of bytes received as response body without
  581. inflation.
  582. URI: request URI
  583. see http://www.w3.org/TR/resource-timing/#processing-model
  584. sorted by 'complete'
  585. id responseEnd requestStart process code size request path
  586. 13 +37.19ms +280us 36.91ms 200 2K /
  587. 2 +72.65ms * +36.38ms 36.26ms 200 8K /stylesheets/screen.css
  588. 17 +77.43ms +38.67ms 38.75ms 200 3K /javascripts/octopress.js
  589. 15 +78.12ms +38.66ms 39.46ms 200 3K /javascripts/modernizr-2.0.js
  590. Using the ``-r`` option, ``nghttp`` writes more detailed timing data to
  591. the given file in HAR format.
  592. nghttpd - server
  593. ++++++++++++++++
  594. ``nghttpd`` is a multi-threaded static web server.
  595. By default, it uses SSL/TLS connection. Use ``--no-tls`` option to
  596. disable it.
  597. ``nghttpd`` only accepts HTTP/2 connections via NPN/ALPN or direct
  598. HTTP/2 connections. No HTTP Upgrade is supported.
  599. The ``-p`` option allows users to configure server push.
  600. Just like ``nghttp``, it has a verbose output mode for framing
  601. information. Here is sample output from ``nghttpd``:
  602. .. code-block:: text
  603. $ nghttpd --no-tls -v 8080
  604. IPv4: listen 0.0.0.0:8080
  605. IPv6: listen :::8080
  606. [id=1] [ 1.521] send SETTINGS frame <length=6, flags=0x00, stream_id=0>
  607. (niv=1)
  608. [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
  609. [id=1] [ 1.521] recv SETTINGS frame <length=12, flags=0x00, stream_id=0>
  610. (niv=2)
  611. [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
  612. [SETTINGS_INITIAL_WINDOW_SIZE(0x04):65535]
  613. [id=1] [ 1.521] recv SETTINGS frame <length=0, flags=0x01, stream_id=0>
  614. ; ACK
  615. (niv=0)
  616. [id=1] [ 1.521] recv PRIORITY frame <length=5, flags=0x00, stream_id=3>
  617. (dep_stream_id=0, weight=201, exclusive=0)
  618. [id=1] [ 1.521] recv PRIORITY frame <length=5, flags=0x00, stream_id=5>
  619. (dep_stream_id=0, weight=101, exclusive=0)
  620. [id=1] [ 1.521] recv PRIORITY frame <length=5, flags=0x00, stream_id=7>
  621. (dep_stream_id=0, weight=1, exclusive=0)
  622. [id=1] [ 1.521] recv PRIORITY frame <length=5, flags=0x00, stream_id=9>
  623. (dep_stream_id=7, weight=1, exclusive=0)
  624. [id=1] [ 1.521] recv PRIORITY frame <length=5, flags=0x00, stream_id=11>
  625. (dep_stream_id=3, weight=1, exclusive=0)
  626. [id=1] [ 1.521] recv (stream_id=13) :method: GET
  627. [id=1] [ 1.521] recv (stream_id=13) :path: /
  628. [id=1] [ 1.521] recv (stream_id=13) :scheme: http
  629. [id=1] [ 1.521] recv (stream_id=13) :authority: localhost:8080
  630. [id=1] [ 1.521] recv (stream_id=13) accept: */*
  631. [id=1] [ 1.521] recv (stream_id=13) accept-encoding: gzip, deflate
  632. [id=1] [ 1.521] recv (stream_id=13) user-agent: nghttp2/1.0.0-DEV
  633. [id=1] [ 1.521] recv HEADERS frame <length=41, flags=0x25, stream_id=13>
  634. ; END_STREAM | END_HEADERS | PRIORITY
  635. (padlen=0, dep_stream_id=11, weight=16, exclusive=0)
  636. ; Open new stream
  637. [id=1] [ 1.521] send SETTINGS frame <length=0, flags=0x01, stream_id=0>
  638. ; ACK
  639. (niv=0)
  640. [id=1] [ 1.521] send HEADERS frame <length=86, flags=0x04, stream_id=13>
  641. ; END_HEADERS
  642. (padlen=0)
  643. ; First response header
  644. :status: 200
  645. server: nghttpd nghttp2/1.0.0-DEV
  646. content-length: 10
  647. cache-control: max-age=3600
  648. date: Fri, 15 May 2015 14:49:04 GMT
  649. last-modified: Tue, 30 Sep 2014 12:40:52 GMT
  650. [id=1] [ 1.522] send DATA frame <length=10, flags=0x01, stream_id=13>
  651. ; END_STREAM
  652. [id=1] [ 1.522] stream_id=13 closed
  653. [id=1] [ 1.522] recv GOAWAY frame <length=8, flags=0x00, stream_id=0>
  654. (last_stream_id=0, error_code=NO_ERROR(0x00), opaque_data(0)=[])
  655. [id=1] [ 1.522] closed
  656. nghttpx - proxy
  657. +++++++++++++++
  658. ``nghttpx`` is a multi-threaded reverse proxy for HTTP/3, HTTP/2, and
  659. HTTP/1.1, and powers http://nghttp2.org and supports HTTP/2 server
  660. push.
  661. We reworked ``nghttpx`` command-line interface, and as a result, there
  662. are several incompatibles from 1.8.0 or earlier. This is necessary to
  663. extend its capability, and secure the further feature enhancements in
  664. the future release. Please read `Migration from nghttpx v1.8.0 or
  665. earlier
  666. <https://nghttp2.org/documentation/nghttpx-howto.html#migration-from-nghttpx-v1-8-0-or-earlier>`_
  667. to know how to migrate from earlier releases.
  668. ``nghttpx`` implements `important performance-oriented features
  669. <https://istlsfastyet.com/#server-performance>`_ in TLS, such as
  670. session IDs, session tickets (with automatic key rotation), OCSP
  671. stapling, dynamic record sizing, ALPN/NPN, forward secrecy and HTTP/2.
  672. ``nghttpx`` also offers the functionality to share session cache and
  673. ticket keys among multiple ``nghttpx`` instances via memcached.
  674. ``nghttpx`` has 2 operation modes:
  675. ================== ======================== ================ =============
  676. Mode option Frontend Backend Note
  677. ================== ======================== ================ =============
  678. default mode HTTP/3, HTTP/2, HTTP/1.1 HTTP/1.1, HTTP/2 Reverse proxy
  679. ``--http2-proxy`` HTTP/3, HTTP/2, HTTP/1.1 HTTP/1.1, HTTP/2 Forward proxy
  680. ================== ======================== ================ =============
  681. The interesting mode at the moment is the default mode. It works like
  682. a reverse proxy and listens for HTTP/3, HTTP/2, and HTTP/1.1 and can
  683. be deployed as a SSL/TLS terminator for existing web server.
  684. In all modes, the frontend connections are encrypted by SSL/TLS by
  685. default. To disable encryption, use the ``no-tls`` keyword in
  686. ``--frontend`` option. If encryption is disabled, incoming HTTP/1.1
  687. connections can be upgraded to HTTP/2 through HTTP Upgrade. On the
  688. other hard, backend connections are not encrypted by default. To
  689. encrypt backend connections, use ``tls`` keyword in ``--backend``
  690. option.
  691. ``nghttpx`` supports a configuration file. See the ``--conf`` option and
  692. sample configuration file ``nghttpx.conf.sample``.
  693. In the default mode, ``nghttpx`` works as reverse proxy to the backend
  694. server:
  695. .. code-block:: text
  696. Client <-- (HTTP/3, HTTP/2, HTTP/1.1) --> nghttpx <-- (HTTP/1.1, HTTP/2) --> Web Server
  697. [reverse proxy]
  698. With the ``--http2-proxy`` option, it works as forward proxy, and it
  699. is so called secure HTTP/2 proxy:
  700. .. code-block:: text
  701. Client <-- (HTTP/3, HTTP/2, HTTP/1.1) --> nghttpx <-- (HTTP/1.1) --> Proxy
  702. [secure proxy] (e.g., Squid, ATS)
  703. The ``Client`` in the above example needs to be configured to use
  704. ``nghttpx`` as secure proxy.
  705. At the time of this writing, both Chrome and Firefox support secure
  706. HTTP/2 proxy. One way to configure Chrome to use a secure proxy is to
  707. create a proxy.pac script like this:
  708. .. code-block:: javascript
  709. function FindProxyForURL(url, host) {
  710. return "HTTPS SERVERADDR:PORT";
  711. }
  712. ``SERVERADDR`` and ``PORT`` is the hostname/address and port of the
  713. machine nghttpx is running on. Please note that Chrome requires a valid
  714. certificate for secure proxy.
  715. Then run Chrome with the following arguments:
  716. .. code-block:: text
  717. $ google-chrome --proxy-pac-url=file:///path/to/proxy.pac --use-npn
  718. The backend HTTP/2 connections can be tunneled through an HTTP proxy.
  719. The proxy is specified using ``--backend-http-proxy-uri``. The
  720. following figure illustrates how nghttpx talks to the outside HTTP/2
  721. proxy through an HTTP proxy:
  722. .. code-block:: text
  723. Client <-- (HTTP/3, HTTP/2, HTTP/1.1) --> nghttpx <-- (HTTP/2) --
  724. --===================---> HTTP/2 Proxy
  725. (HTTP proxy tunnel) (e.g., nghttpx -s)
  726. Benchmarking tool
  727. -----------------
  728. The ``h2load`` program is a benchmarking tool for HTTP/3, HTTP/2, and
  729. HTTP/1.1. The UI of ``h2load`` is heavily inspired by ``weighttp``
  730. (https://github.com/lighttpd/weighttp). The typical usage is as
  731. follows:
  732. .. code-block:: text
  733. $ h2load -n100000 -c100 -m100 https://localhost:8443/
  734. starting benchmark...
  735. spawning thread #0: 100 concurrent clients, 100000 total requests
  736. Protocol: TLSv1.2
  737. Cipher: ECDHE-RSA-AES128-GCM-SHA256
  738. Server Temp Key: ECDH P-256 256 bits
  739. progress: 10% done
  740. progress: 20% done
  741. progress: 30% done
  742. progress: 40% done
  743. progress: 50% done
  744. progress: 60% done
  745. progress: 70% done
  746. progress: 80% done
  747. progress: 90% done
  748. progress: 100% done
  749. finished in 771.26ms, 129658 req/s, 4.71MB/s
  750. requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored
  751. status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
  752. traffic: 3812300 bytes total, 1009900 bytes headers, 1000000 bytes data
  753. min max mean sd +/- sd
  754. time for request: 25.12ms 124.55ms 51.07ms 15.36ms 84.87%
  755. time for connect: 208.94ms 254.67ms 241.38ms 7.95ms 63.00%
  756. time to 1st byte: 209.11ms 254.80ms 241.51ms 7.94ms 63.00%
  757. The above example issued total 100,000 requests, using 100 concurrent
  758. clients (in other words, 100 HTTP/2 sessions), and a maximum of 100 streams
  759. per client. With the ``-t`` option, ``h2load`` will use multiple native
  760. threads to avoid saturating a single core on client side.
  761. .. warning::
  762. **Don't use this tool against publicly available servers.** That is
  763. considered a DOS attack. Please only use it against your private
  764. servers.
  765. If the experimental HTTP/3 is enabled, h2load can send requests to
  766. HTTP/3 server. To do this, specify ``h3`` to ``--npn-list`` option
  767. like so:
  768. .. code-block:: text
  769. $ h2load --npn-list h3 https://127.0.0.1:4433
  770. HPACK tools
  771. -----------
  772. The ``src`` directory contains the HPACK tools. The ``deflatehd`` program is a
  773. command-line header compression tool. The ``inflatehd`` program is a
  774. command-line header decompression tool. Both tools read input from
  775. stdin and write output to stdout. Errors are written to stderr.
  776. They take JSON as input and output. We (mostly) use the same JSON data
  777. format described at https://github.com/http2jp/hpack-test-case.
  778. deflatehd - header compressor
  779. +++++++++++++++++++++++++++++
  780. The ``deflatehd`` program reads JSON data or HTTP/1-style header fields from
  781. stdin and outputs compressed header block in JSON.
  782. For the JSON input, the root JSON object must include a ``cases`` key.
  783. Its value has to include the sequence of input header set. They share
  784. the same compression context and are processed in the order they
  785. appear. Each item in the sequence is a JSON object and it must
  786. include a ``headers`` key. Its value is an array of JSON objects,
  787. which includes exactly one name/value pair.
  788. Example:
  789. .. code-block:: json
  790. {
  791. "cases":
  792. [
  793. {
  794. "headers": [
  795. { ":method": "GET" },
  796. { ":path": "/" }
  797. ]
  798. },
  799. {
  800. "headers": [
  801. { ":method": "POST" },
  802. { ":path": "/" }
  803. ]
  804. }
  805. ]
  806. }
  807. With the ``-t`` option, the program can accept more familiar HTTP/1 style
  808. header field blocks. Each header set is delimited by an empty line:
  809. Example:
  810. .. code-block:: text
  811. :method: GET
  812. :scheme: https
  813. :path: /
  814. :method: POST
  815. user-agent: nghttp2
  816. The output is in JSON object. It should include a ``cases`` key and its
  817. value is an array of JSON objects, which has at least the following keys:
  818. seq
  819. The index of header set in the input.
  820. input_length
  821. The sum of the length of the name/value pairs in the input.
  822. output_length
  823. The length of the compressed header block.
  824. percentage_of_original_size
  825. ``output_length`` / ``input_length`` * 100
  826. wire
  827. The compressed header block as a hex string.
  828. headers
  829. The input header set.
  830. header_table_size
  831. The header table size adjusted before deflating the header set.
  832. Examples:
  833. .. code-block:: json
  834. {
  835. "cases":
  836. [
  837. {
  838. "seq": 0,
  839. "input_length": 66,
  840. "output_length": 20,
  841. "percentage_of_original_size": 30.303030303030305,
  842. "wire": "01881f3468e5891afcbf83868a3d856659c62e3f",
  843. "headers": [
  844. {
  845. ":authority": "example.org"
  846. },
  847. {
  848. ":method": "GET"
  849. },
  850. {
  851. ":path": "/"
  852. },
  853. {
  854. ":scheme": "https"
  855. },
  856. {
  857. "user-agent": "nghttp2"
  858. }
  859. ],
  860. "header_table_size": 4096
  861. }
  862. ,
  863. {
  864. "seq": 1,
  865. "input_length": 74,
  866. "output_length": 10,
  867. "percentage_of_original_size": 13.513513513513514,
  868. "wire": "88448504252dd5918485",
  869. "headers": [
  870. {
  871. ":authority": "example.org"
  872. },
  873. {
  874. ":method": "POST"
  875. },
  876. {
  877. ":path": "/account"
  878. },
  879. {
  880. ":scheme": "https"
  881. },
  882. {
  883. "user-agent": "nghttp2"
  884. }
  885. ],
  886. "header_table_size": 4096
  887. }
  888. ]
  889. }
  890. The output can be used as the input for ``inflatehd`` and
  891. ``deflatehd``.
  892. With the ``-d`` option, the extra ``header_table`` key is added and its
  893. associated value includes the state of dynamic header table after the
  894. corresponding header set was processed. The value includes at least
  895. the following keys:
  896. entries
  897. The entry in the header table. If ``referenced`` is ``true``, it
  898. is in the reference set. The ``size`` includes the overhead (32
  899. bytes). The ``index`` corresponds to the index of header table.
  900. The ``name`` is the header field name and the ``value`` is the
  901. header field value.
  902. size
  903. The sum of the spaces entries occupied, this includes the
  904. entry overhead.
  905. max_size
  906. The maximum header table size.
  907. deflate_size
  908. The sum of the spaces entries occupied within
  909. ``max_deflate_size``.
  910. max_deflate_size
  911. The maximum header table size the encoder uses. This can be smaller
  912. than ``max_size``. In this case, the encoder only uses up to first
  913. ``max_deflate_size`` buffer. Since the header table size is still
  914. ``max_size``, the encoder has to keep track of entries outside the
  915. ``max_deflate_size`` but inside the ``max_size`` and make sure
  916. that they are no longer referenced.
  917. Example:
  918. .. code-block:: json
  919. {
  920. "cases":
  921. [
  922. {
  923. "seq": 0,
  924. "input_length": 66,
  925. "output_length": 20,
  926. "percentage_of_original_size": 30.303030303030305,
  927. "wire": "01881f3468e5891afcbf83868a3d856659c62e3f",
  928. "headers": [
  929. {
  930. ":authority": "example.org"
  931. },
  932. {
  933. ":method": "GET"
  934. },
  935. {
  936. ":path": "/"
  937. },
  938. {
  939. ":scheme": "https"
  940. },
  941. {
  942. "user-agent": "nghttp2"
  943. }
  944. ],
  945. "header_table_size": 4096,
  946. "header_table": {
  947. "entries": [
  948. {
  949. "index": 1,
  950. "name": "user-agent",
  951. "value": "nghttp2",
  952. "referenced": true,
  953. "size": 49
  954. },
  955. {
  956. "index": 2,
  957. "name": ":scheme",
  958. "value": "https",
  959. "referenced": true,
  960. "size": 44
  961. },
  962. {
  963. "index": 3,
  964. "name": ":path",
  965. "value": "/",
  966. "referenced": true,
  967. "size": 38
  968. },
  969. {
  970. "index": 4,
  971. "name": ":method",
  972. "value": "GET",
  973. "referenced": true,
  974. "size": 42
  975. },
  976. {
  977. "index": 5,
  978. "name": ":authority",
  979. "value": "example.org",
  980. "referenced": true,
  981. "size": 53
  982. }
  983. ],
  984. "size": 226,
  985. "max_size": 4096,
  986. "deflate_size": 226,
  987. "max_deflate_size": 4096
  988. }
  989. }
  990. ,
  991. {
  992. "seq": 1,
  993. "input_length": 74,
  994. "output_length": 10,
  995. "percentage_of_original_size": 13.513513513513514,
  996. "wire": "88448504252dd5918485",
  997. "headers": [
  998. {
  999. ":authority": "example.org"
  1000. },
  1001. {
  1002. ":method": "POST"
  1003. },
  1004. {
  1005. ":path": "/account"
  1006. },
  1007. {
  1008. ":scheme": "https"
  1009. },
  1010. {
  1011. "user-agent": "nghttp2"
  1012. }
  1013. ],
  1014. "header_table_size": 4096,
  1015. "header_table": {
  1016. "entries": [
  1017. {
  1018. "index": 1,
  1019. "name": ":method",
  1020. "value": "POST",
  1021. "referenced": true,
  1022. "size": 43
  1023. },
  1024. {
  1025. "index": 2,
  1026. "name": "user-agent",
  1027. "value": "nghttp2",
  1028. "referenced": true,
  1029. "size": 49
  1030. },
  1031. {
  1032. "index": 3,
  1033. "name": ":scheme",
  1034. "value": "https",
  1035. "referenced": true,
  1036. "size": 44
  1037. },
  1038. {
  1039. "index": 4,
  1040. "name": ":path",
  1041. "value": "/",
  1042. "referenced": false,
  1043. "size": 38
  1044. },
  1045. {
  1046. "index": 5,
  1047. "name": ":method",
  1048. "value": "GET",
  1049. "referenced": false,
  1050. "size": 42
  1051. },
  1052. {
  1053. "index": 6,
  1054. "name": ":authority",
  1055. "value": "example.org",
  1056. "referenced": true,
  1057. "size": 53
  1058. }
  1059. ],
  1060. "size": 269,
  1061. "max_size": 4096,
  1062. "deflate_size": 269,
  1063. "max_deflate_size": 4096
  1064. }
  1065. }
  1066. ]
  1067. }
  1068. inflatehd - header decompressor
  1069. +++++++++++++++++++++++++++++++
  1070. The ``inflatehd`` program reads JSON data from stdin and outputs decompressed
  1071. name/value pairs in JSON.
  1072. The root JSON object must include the ``cases`` key. Its value has to
  1073. include the sequence of compressed header blocks. They share the same
  1074. compression context and are processed in the order they appear. Each
  1075. item in the sequence is a JSON object and it must have at least a
  1076. ``wire`` key. Its value is a compressed header block as a hex string.
  1077. Example:
  1078. .. code-block:: json
  1079. {
  1080. "cases":
  1081. [
  1082. { "wire": "8285" },
  1083. { "wire": "8583" }
  1084. ]
  1085. }
  1086. The output is a JSON object. It should include a ``cases`` key and its
  1087. value is an array of JSON objects, which has at least following keys:
  1088. seq
  1089. The index of the header set in the input.
  1090. headers
  1091. A JSON array that includes decompressed name/value pairs.
  1092. wire
  1093. The compressed header block as a hex string.
  1094. header_table_size
  1095. The header table size adjusted before inflating compressed header
  1096. block.
  1097. Example:
  1098. .. code-block:: json
  1099. {
  1100. "cases":
  1101. [
  1102. {
  1103. "seq": 0,
  1104. "wire": "01881f3468e5891afcbf83868a3d856659c62e3f",
  1105. "headers": [
  1106. {
  1107. ":authority": "example.org"
  1108. },
  1109. {
  1110. ":method": "GET"
  1111. },
  1112. {
  1113. ":path": "/"
  1114. },
  1115. {
  1116. ":scheme": "https"
  1117. },
  1118. {
  1119. "user-agent": "nghttp2"
  1120. }
  1121. ],
  1122. "header_table_size": 4096
  1123. }
  1124. ,
  1125. {
  1126. "seq": 1,
  1127. "wire": "88448504252dd5918485",
  1128. "headers": [
  1129. {
  1130. ":method": "POST"
  1131. },
  1132. {
  1133. ":path": "/account"
  1134. },
  1135. {
  1136. "user-agent": "nghttp2"
  1137. },
  1138. {
  1139. ":scheme": "https"
  1140. },
  1141. {
  1142. ":authority": "example.org"
  1143. }
  1144. ],
  1145. "header_table_size": 4096
  1146. }
  1147. ]
  1148. }
  1149. The output can be used as the input for ``deflatehd`` and
  1150. ``inflatehd``.
  1151. With the ``-d`` option, the extra ``header_table`` key is added and its
  1152. associated value includes the state of the dynamic header table after the
  1153. corresponding header set was processed. The format is the same as
  1154. ``deflatehd``.
  1155. libnghttp2_asio: High level HTTP/2 C++ library
  1156. ----------------------------------------------
  1157. libnghttp2_asio is C++ library built on top of libnghttp2 and provides
  1158. high level abstraction API to build HTTP/2 applications. It depends
  1159. on the Boost::ASIO library and OpenSSL. Currently libnghttp2_asio
  1160. provides both client and server APIs.
  1161. libnghttp2_asio is not built by default. Use the ``--enable-asio-lib``
  1162. configure flag to build libnghttp2_asio. The required Boost libraries
  1163. are:
  1164. * Boost::Asio
  1165. * Boost::System
  1166. * Boost::Thread
  1167. The server API is designed to build an HTTP/2 server very easily to utilize
  1168. C++14 anonymous functions and closures. The bare minimum example of
  1169. an HTTP/2 server looks like this:
  1170. .. code-block:: cpp
  1171. #include <iostream>
  1172. #include <nghttp2/asio_http2_server.h>
  1173. using namespace nghttp2::asio_http2;
  1174. using namespace nghttp2::asio_http2::server;
  1175. int main(int argc, char *argv[]) {
  1176. boost::system::error_code ec;
  1177. http2 server;
  1178. server.handle("/", [](const request &req, const response &res) {
  1179. res.write_head(200);
  1180. res.end("hello, world\n");
  1181. });
  1182. if (server.listen_and_serve(ec, "localhost", "3000")) {
  1183. std::cerr << "error: " << ec.message() << std::endl;
  1184. }
  1185. }
  1186. Here is sample code to use the client API:
  1187. .. code-block:: cpp
  1188. #include <iostream>
  1189. #include <nghttp2/asio_http2_client.h>
  1190. using boost::asio::ip::tcp;
  1191. using namespace nghttp2::asio_http2;
  1192. using namespace nghttp2::asio_http2::client;
  1193. int main(int argc, char *argv[]) {
  1194. boost::system::error_code ec;
  1195. boost::asio::io_service io_service;
  1196. // connect to localhost:3000
  1197. session sess(io_service, "localhost", "3000");
  1198. sess.on_connect([&sess](tcp::resolver::iterator endpoint_it) {
  1199. boost::system::error_code ec;
  1200. auto req = sess.submit(ec, "GET", "http://localhost:3000/");
  1201. req->on_response([](const response &res) {
  1202. // print status code and response header fields.
  1203. std::cerr << "HTTP/2 " << res.status_code() << std::endl;
  1204. for (auto &kv : res.header()) {
  1205. std::cerr << kv.first << ": " << kv.second.value << "\n";
  1206. }
  1207. std::cerr << std::endl;
  1208. res.on_data([](const uint8_t *data, std::size_t len) {
  1209. std::cerr.write(reinterpret_cast<const char *>(data), len);
  1210. std::cerr << std::endl;
  1211. });
  1212. });
  1213. req->on_close([&sess](uint32_t error_code) {
  1214. // shutdown session after first request was done.
  1215. sess.shutdown();
  1216. });
  1217. });
  1218. sess.on_error([](const boost::system::error_code &ec) {
  1219. std::cerr << "error: " << ec.message() << std::endl;
  1220. });
  1221. io_service.run();
  1222. }
  1223. For more details, see the documentation of libnghttp2_asio.
  1224. Python bindings
  1225. ---------------
  1226. The ``python`` directory contains nghttp2 Python bindings. The
  1227. bindings currently provide HPACK compressor and decompressor classes
  1228. and an HTTP/2 server.
  1229. The extension module is called ``nghttp2``.
  1230. ``make`` will build the bindings and target Python version is
  1231. determined by the ``configure`` script. If the detected Python version is not
  1232. what you expect, specify a path to Python executable in a ``PYTHON``
  1233. variable as an argument to configure script (e.g., ``./configure
  1234. PYTHON=/usr/bin/python3.8``).
  1235. The following example code illustrates basic usage of the HPACK compressor
  1236. and decompressor in Python:
  1237. .. code-block:: python
  1238. import binascii
  1239. import nghttp2
  1240. deflater = nghttp2.HDDeflater()
  1241. inflater = nghttp2.HDInflater()
  1242. data = deflater.deflate([(b'foo', b'bar'),
  1243. (b'baz', b'buz')])
  1244. print(binascii.b2a_hex(data))
  1245. hdrs = inflater.inflate(data)
  1246. print(hdrs)
  1247. The ``nghttp2.HTTP2Server`` class builds on top of the asyncio event
  1248. loop. On construction, *RequestHandlerClass* must be given, which
  1249. must be a subclass of ``nghttp2.BaseRequestHandler`` class.
  1250. The ``BaseRequestHandler`` class is used to handle the HTTP/2 stream.
  1251. By default, it does nothing. It must be subclassed to handle each
  1252. event callback method.
  1253. The first callback method invoked is ``on_headers()``. It is called
  1254. when HEADERS frame, which includes the request header fields, has arrived.
  1255. If the request has a request body, ``on_data(data)`` is invoked for each
  1256. chunk of received data.
  1257. Once the entire request is received, ``on_request_done()`` is invoked.
  1258. When the stream is closed, ``on_close(error_code)`` is called.
  1259. The application can send a response using ``send_response()`` method.
  1260. It can be used in ``on_headers()``, ``on_data()`` or
  1261. ``on_request_done()``.
  1262. The application can push resources using the ``push()`` method. It must be
  1263. used before the ``send_response()`` call.
  1264. The following instance variables are available:
  1265. client_address
  1266. Contains a tuple of the form (host, port) referring to the
  1267. client's address.
  1268. stream_id
  1269. Stream ID of this stream.
  1270. scheme
  1271. Scheme of the request URI. This is a value of :scheme header
  1272. field.
  1273. method
  1274. Method of this stream. This is a value of :method header field.
  1275. host
  1276. This is a value of :authority or host header field.
  1277. path
  1278. This is a value of :path header field.
  1279. The following example illustrates the HTTP2Server and
  1280. BaseRequestHandler usage:
  1281. .. code-block:: python
  1282. #!/usr/bin/env python3
  1283. import io, ssl
  1284. import nghttp2
  1285. class Handler(nghttp2.BaseRequestHandler):
  1286. def on_headers(self):
  1287. self.push(path='/css/bootstrap.css',
  1288. request_headers = [('content-length', '3')],
  1289. status=200,
  1290. body='foo')
  1291. self.push(path='/js/bootstrap.js',
  1292. method='GET',
  1293. request_headers = [('content-length', '10')],
  1294. status=200,
  1295. body='foobarbuzz')
  1296. self.send_response(status=200,
  1297. headers = [('content-type', 'text/plain')],
  1298. body=io.BytesIO(b'nghttp2-python FTW'))
  1299. ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
  1300. ctx.options = ssl.OP_ALL | ssl.OP_NO_SSLv2
  1301. ctx.load_cert_chain('server.crt', 'server.key')
  1302. # give None to ssl to make the server non-SSL/TLS
  1303. server = nghttp2.HTTP2Server(('127.0.0.1', 8443), Handler, ssl=ctx)
  1304. server.serve_forever()
  1305. Contribution
  1306. ------------
  1307. [This text was composed based on 1.2. License section of curl/libcurl
  1308. project.]
  1309. When contributing with code, you agree to put your changes and new
  1310. code under the same license nghttp2 is already using unless stated and
  1311. agreed otherwise.
  1312. When changing existing source code, do not alter the copyright of
  1313. the original file(s). The copyright will still be owned by the
  1314. original creator(s) or those who have been assigned copyright by the
  1315. original author(s).
  1316. By submitting a patch to the nghttp2 project, you (or your employer, as
  1317. the case may be) agree to assign the copyright of your submission to us.
  1318. .. the above really needs to be reworded to pass legal muster.
  1319. We will credit you for your
  1320. changes as far as possible, to give credit but also to keep a trace
  1321. back to who made what changes. Please always provide us with your
  1322. full real name when contributing!
  1323. See `Contribution Guidelines
  1324. <https://nghttp2.org/documentation/contribute.html>`_ for more
  1325. details.
  1326. Reporting vulnerability
  1327. -----------------------
  1328. If you find a vulnerability in our software, please send the email to
  1329. "tatsuhiro.t at gmail dot com" about its details instead of submitting
  1330. issues on github issue page. It is a standard practice not to
  1331. disclose vulnerability information publicly until a fixed version is
  1332. released, or mitigation is worked out.
  1333. In the future, we may setup a dedicated mail address for this purpose.
  1334. Release schedule
  1335. ----------------
  1336. In general, we follow `Semantic Versioning <http://semver.org/>`_. We
  1337. release MINOR version update every month, and usually we ship it
  1338. around 25th day of every month.
  1339. We may release PATCH releases between the regular releases, mainly for
  1340. severe security bug fixes.
  1341. We have no plan to break API compatibility changes involving soname
  1342. bump, so MAJOR version will stay 1 for the foreseeable future.
  1343. License
  1344. -------
  1345. The MIT License