Changelog for the libssh2 project. Generated with git2news.pl Daniel Stenberg (29 Aug 2021) - [Will Cosgrove brought this change] updated docs for 1.10.0 release Marc Hörsken (30 May 2021) - [Laurent Stacul brought this change] [tests] Try several times to connect the ssh server Sometimes, as the OCI container is run in detached mode, it is possible the actual server is not ready yet to handle SSH traffic. The goal of this PR is to try several times (max 3). The mechanism is the same as for the connection to the docker machine. - [Laurent Stacul brought this change] Remove openssh_server container on test exit - [Laurent Stacul brought this change] Allow the tests to run inside a container The current tests suite starts SSH server as OCI container. This commit add the possibility to run the tests in a container provided that: * the docker client is installed builder container * the host docker daemon unix socket has been mounted in the builder container (with, if needed, the DOCKER_HOST environment variable accordingly set, and the permission to write on this socket) * the builder container is run on the default bridge network, or the host network. This PR does not handle the case where the builder container is on another network. Marc Hoersken (28 May 2021) - CI/appveyor: run SSH server for tests on GitHub Actions (#607) No longer rely on DigitalOcean to host the Docker container. Unfortunately we require a small dispatcher script that has access to a GitHub access token with scope repo in order to trigger the daemon workflow on GitHub Actions also for PRs. This script is hosted by myself for the time being until GitHub provides a tighter scope to trigger the workflow_dispatch event. GitHub (26 May 2021) - [Will Cosgrove brought this change] openssl.c: guards around calling FIPS_mode() #596 (#603) Notes: FIPS_mode() is not implemented in LibreSSL and this API is removed in OpenSSL 3.0 and was introduced in 0.9.7. Added guards around making this call. Credit: Will Cosgrove - [Will Cosgrove brought this change] configure.ac: don't undefine scoped variable (#594) * configure.ac: don't undefine scoped variable To get this script to run with Autoconf 2.71 on macOS I had to remove the undefine of the backend for loop variable. It seems scoped to the for loop and also isn't referenced later in the script so it seems OK to remove it. * configure.ac: remove cygwin specific CFLAGS #598 Notes: Remove cygwin specific Win32 CFLAGS and treat the build like a posix build Credit: Will Cosgrove, Brian Inglis - [Laurent Stacul brought this change] tests: Makefile.am: Add missing tests client keys in distribution tarball (#604) Notes: Added missing test keys. Credit: Laurent Stacul - [Laurent Stacul brought this change] Makefile.am: Add missing test keys in the distribution tarball (#601) Notes: Fix tests missing key to build the OCI image Credit: Laurent Stacul Daniel Stenberg (16 May 2021) - dist: add src/agent.h Fixes #597 Closes #599 GitHub (12 May 2021) - [Will Cosgrove brought this change] packet.c: Reset read timeout after received a packet (#576) (#586) File: packet.c Notes: Attempt keyboard interactive login (Azure AD 2FA login) and use more than 60 seconds to complete the login, the connection fails. The _libssh2_packet_require function does almost the same as _libssh2_packet_requirev but this function sets state->start = 0 before returning. Credit: teottin, Co-authored-by: Tor Erik Ottinsen <tor.ottinsen@kdi.kongsberg.com> - [kkoenig brought this change] Support ECDSA certificate authentication (#570) Files: hostkey.c, userauth.c, test_public_key_auth_succeeds_with_correct_ecdsa_key.c Notes: Support ECDSA certificate authentication Add a test for: - Existing ecdsa basic public key authentication - ecdsa public key authentication with a signed public key Credit: kkoenig - [Gabriel Smith brought this change] agent.c: Add support for Windows OpenSSH agent (#517) Files: agent.c, agent.h, agent_win.c Notes: * agent: Add support for Windows OpenSSH agent The implementation was partially taken and modified from that found in the Portable OpenSSH port to Win32 by the PowerShell team, but mostly based on the existing Unix OpenSSH agent support. https://github.com/PowerShell/openssh-portable Regarding the partial transfer support implementation: partial transfers are easy to deal with, but you need to track additional state when non-blocking IO enters the picture. A tracker of how many bytes have been transfered has been placed in the transfer context struct as that's where it makes most sense. This tracker isn't placed behind a WIN32 #ifdef as it will probably be useful for other agent implementations. * agent: win32 openssh: Disable overlapped IO Non-blocking IO is not currently supported by the surrounding agent code, despite a lot of the code having everything set up to handle it. Credit: Co-authored-by: Gabriel Smith <gabriel.smith@precisionot.com> - [Zenju brought this change] Fix detailed _libssh2_error being overwritten (#473) Files: openssl.c, pem.c, userauth.c Notes: * Fix detailed _libssh2_error being overwritten by generic errors * Unified error handling Credit: Zenju - [Paul Capron brought this change] Fix _libssh2_random() silently discarding errors (#520) Notes: * Make _libssh2_random return code consistent Previously, _libssh2_random was advertized in HACKING.CRYPTO as returning `void` (and was implemented that way in os400qc3.c), but that was in other crypto backends a lie; _libssh2_random is (a macro expanding) to an int-value expression or function. Moreover, that returned code was: — 0 or success, -1 on error for the MbedTLS & WinCNG crypto backends But also: — 1 on success, -1 or 0 on error for the OpenSSL backend! – 1 on success, error cannot happen for libgcrypt! This commit makes explicit that _libssh2_random can fail (because most of the underlying crypto functions can indeed fail!), and it makes its result code consistent: 0 on success, -1 on error. This is related to issue #519 https://github.com/libssh2/libssh2/issues/519 It fixes the first half of it. * Don't silent errors of _libssh2_random Make sure to check the returned code of _libssh2_random(), and propagates any failure. A new LIBSSH_ERROR_RANDGEN constant is added to libssh2.h None of the existing error constants seemed fit. This commit is related to d74285b68450c0e9ea6d5f8070450837fb1e74a7 and to https://github.com/libssh2/libssh2/issues/519 (see the issue for more info.) It closes #519. Credit: Paul Capron - [Gabriel Smith brought this change] ci: Remove caching of docker image layers (#589) Notes: continued ci reliability work. Credit: Gabriel Smith - [Gabriel Smith brought this change] ci: Speed up docker builds for tests (#587) Notes: The OpenSSH server docker image used for tests is pre-built to prevent wasting time building it during a test, and unneeded rebuilds are prevented by caching the image layers. Credit: Gabriel Smith - [Will Cosgrove brought this change] userauth.c: don't error if using keys without RSA (#555) file: userauth.c notes: libssh2 now supports many other key types besides RSA, if the library is built without RSA support and a user attempts RSA auth it shouldn't be an automatic error credit: Will Cosgrove - [Marc brought this change] openssl.c: Avoid OpenSSL latent error in FIPS mode (#528) File: openssl.c Notes: Avoid initing MD5 digest, which is not permitted in OpenSSL FIPS certified cryptography mode. Credit: Marc - [Laurent Stacul brought this change] openssl.c: Fix EVP_Cipher interface change in openssl 3 #463 File: openssl.c Notes: Fixes building with OpenSSL 3, #463. The change is described there: https://github.com/openssl/openssl/commit/f7397f0d58ce7ddf4c5366cd1846f16b341fbe43 Credit: Laurent Stacul, reported by Sergei - [Gabriel Smith brought this change] openssh_fixture.c: Fix potential overwrite of buffer when reading stdout of command (#580) File: openssh_fixture.c Notes: If reading the full output from the executed command took multiple passes (such as when reading multiple lines) the old code would read into the buffer starting at the some position (the start) every time. The old code only works if fgets updated p or had an offset parameter, both of which are not true. Credit: Gabriel Smith - [Gabriel Smith brought this change] ci: explicitly state the default branch (#585) Notes: It looks like the $default-branch macro only works in templates, not workflows. This is not explicitly stated anywhere except the linked PR comment. https://github.com/actions/starter-workflows/pull/590#issuecomment-672360634 credit: Gabriel Smith - [Gabriel Smith brought this change] ci: Swap from Travis to Github Actions (#581) Files: ci files Notes: Move Linux CI using Github Actions Credit: Gabriel Smith, Marc Hörsken - [Mary brought this change] libssh2_priv.h: add iovec on 3ds (#575) file: libssh2_priv.h note: include iovec for 3DS credit: Mary Mstrodl - [Laurent Stacul brought this change] Tests: Fix unused variables warning (#561) file: test_public_key_auth_succeeds_with_correct_ed25519_key_from_mem.c notes: fixed unused vars credit: Laurent Stacul - [Viktor Szakats brought this change] bcrypt_pbkdf.c: fix clang10 false positive warning (#563) File: bcrypt_pbkdf.c Notes: blf_enc() takes a number of 64-bit blocks to encrypt, but using sizeof(uint64_t) in the calculation triggers a warning with clang 10 because the actual data type is uint32_t. Pass BCRYPT_BLOCKS / 2 for the number of blocks like libc bcrypt(3) does. Ref: https://github.com/openbsd/src/commit/04a2240bd8f465bcae6b595d912af3e2965856de Fixes #562 Credit: Viktor Szakats - [Will Cosgrove brought this change] transport.c: release payload on error (#554) file: transport.c notes: If the payload is invalid and there is an early return, we could leak the payload credit: Will Cosgrove - [Will Cosgrove brought this change] ssh2_client_fuzzer.cc: fixed building The GitHub web editor did some funky things - [Will Cosgrove brought this change] ssh_client_fuzzer.cc: set blocking mode on (#553) file: ssh_client_fuzzer.cc notes: the session needs blocking mode turned on to avoid EAGAIN being returned from libssh2_session_handshake() credit: Will Cosgrove, reviewed by Michael Buckley - [Etienne Samson brought this change] Add a LINT option to CMake (#372) * ci: make style-checking available locally * cmake: add a linting target * tests: check test suite syntax with checksrc.pl - [Will Cosgrove brought this change] kex.c: kex_agree_instr() improve string reading (#552) * kex.c: kex_agree_instr() improve string reading file: kex.c notes: if haystack isn't null terminated we should use memchr() not strchar(). We should also make sure we don't walk off the end of the buffer. credit: Will Cosgrove, reviewed by Michael Buckley - [Will Cosgrove brought this change] kex.c: use string_buf in ecdh_sha2_nistp (#551) * kex.c: use string_buf in ecdh_sha2_nistp file: kex.c notes: use string_buf in ecdh_sha2_nistp() to avoid attempting to parse malformed data - [Will Cosgrove brought this change] kex.c: move EC macro outside of if check #549 (#550) File: kex.c Notes: Moved the macro LIBSSH2_KEX_METHOD_EC_SHA_HASH_CREATE_VERIFY outside of the LIBSSH2_ECDSA since it's also now used by the ED25519 code. Sha 256, 384 and 512 need to be defined for all backends now even if they aren't used directly. I believe this is already the case, but just a heads up. Credit: Stefan-Ghinea - [Tim Gates brought this change] kex.c: fix simple typo, niumber -> number (#545) File: kex.c Notes: There is a small typo in src/kex.c. Should read `number` rather than `niumber`. Credit: Tim Gates - [Tseng Jun brought this change] session.c: Correct a typo which may lead to stack overflow (#533) File: session.c Notes: Seems the author intend to terminate banner_dup buffer, later, print it to the debug console. Author: Tseng Jun Marc Hoersken (10 Oct 2020) - wincng: fix random big number generation to match openssl The old function would set the least significant bits in the most significant byte instead of the most significant bits. The old function would also zero pad too much bits in the most significant byte. This lead to a reduction of key space in the most significant byte according to the following listing: - 8 bits reduced to 0 bits => eg. 2048 bits to 2040 bits DH key - 7 bits reduced to 1 bits => eg. 2047 bits to 2041 bits DH key - 6 bits reduced to 2 bits => eg. 2046 bits to 2042 bits DH key - 5 bits reduced to 3 bits => eg. 2045 bits to 2043 bits DH key No change would occur for the case of 4 significant bits. For 1 to 3 significant bits in the most significant byte the DH key would actually be expanded instead of reduced: - 3 bits expanded to 5 bits => eg. 2043 bits to 2045 bits DH key - 2 bits expanded to 6 bits => eg. 2042 bits to 2046 bits DH key - 1 bits expanded to 7 bits => eg. 2041 bits to 2047 bits DH key There is no case of 0 significant bits in the most significant byte since this would be a case of 8 significant bits in the next byte. At the moment only the following case applies due to a fixed DH key size value currently being used in libssh2: The DH group_order is fixed to 256 (bytes) which leads to a 2047 bits DH key size by calculating (256 * 8) - 1. This means the DH keyspace was previously reduced from 2047 bits to 2041 bits (while the top and bottom bits are always set), so the keyspace is actually always reduced from 2045 bits to 2039 bits. All of this is only relevant for Windows versions supporting the WinCNG backend (Vista or newer) before Windows 10 version 1903. Closes #521 Daniel Stenberg (28 Sep 2020) - libssh2_session_callback_set.3: explain the recv/send callbacks Describe how to actually use these callbacks. Closes #518 GitHub (23 Sep 2020) - [Will Cosgrove brought this change] agent.c: formatting Improved formatting of RECV_SEND_ALL macro. - [Will Cosgrove brought this change] CMakeLists.txt: respect install lib dir #405 (#515) Files: CMakeLists.txt Notes: Use CMAKE_INSTALL_LIBDIR directory Credit: Arfrever - [Will Cosgrove brought this change] kex.c: group16-sha512 and group18-sha512 support #457 (#468) Files: kex.c Notes: Added key exchange group16-sha512 and group18-sha512. As a result did the following: Abstracted diffie_hellman_sha256() to diffie_hellman_sha_algo() which is now algorithm agnostic and takes the algorithm as a parameter since we needed sha512 support. Unfortunately it required some helper functions but they are simple. Deleted diffie_hellman_sha1() Deleted diffie_hellman_sha1 specific macro Cleaned up some formatting Defined sha384 in os400 and wincng backends Defined LIBSSH2_DH_MAX_MODULUS_BITS to abort the connection if we receive too large of p from the server doing sha1 key exchange. Reorder the default key exchange list to match OpenSSH and improve security Credit: Will Cosgrove - [Igor Klevanets brought this change] agent.c: Recv and send all bytes via network in agent_transact_unix() (#510) Files: agent.c Notes: Handle sending/receiving partial packet replies in agent.c API. Credit: Klevanets Igor <cerevra@yandex-team.ru> - [Daniel Stenberg brought this change] Makefile.am: include all test files in the dist #379 File: Makefile.am Notes: No longer conditionally include OpenSSL specific test files, they aren't run if we're not building against OpenSSL 1.1.x anyway. Credit: Daniel Stenberg - [Max Dymond brought this change] Add support for an OSS Fuzzer fuzzing target (#392) Files: .travis.yml, configure.ac, ossfuzz Notes: This adds support for an OSS-Fuzz fuzzing target in ssh2_client_fuzzer, which is a cut down example of ssh2.c. Future enhancements can improve coverage. Credit: Max Dymond - [Sebastián Katzer brought this change] mbedtls.c: ECDSA support for mbed TLS (#385) Files: mbedtls.c, mbedtls.h, .travis.yml Notes: This PR adds support for ECDSA for both key exchange and host key algorithms. The following elliptic curves are supported: 256-bit curve defined by FIPS 186-4 and SEC1 384-bit curve defined by FIPS 186-4 and SEC1 521-bit curve defined by FIPS 186-4 and SEC1 Credit: Sebastián Katzer Marc Hoersken (1 Sep 2020) - buildconf: exec autoreconf to avoid additional process (#512) Also make buildconf exit with the return code of autoreconf. Follow up to #224 - scp.c: fix indentation in shell_quotearg documentation - wincng: make more use of new helper functions (#496) - wincng: make sure algorithm providers are closed once (#496) GitHub (10 Jul 2020) - [David Benjamin brought this change] openssl.c: clean up curve25519 code (#499) File: openssl.c, openssl.h, crypto.h, kex.c Notes: This cleans up a few things in the curve25519 implementation: - There is no need to create X509_PUBKEYs or PKCS8_PRIV_KEY_INFOs to extract key material. EVP_PKEY_get_raw_private_key and EVP_PKEY_get_raw_public_key work fine. - libssh2_x25519_ctx was never used (and occasionally mis-typedefed to libssh2_ed25519_ctx). Remove it. The _libssh2_curve25519_new and _libssh2_curve25519_gen_k interfaces use the bytes. Note, if it needs to be added back, there is no need to roundtrip through EVP_PKEY_new_raw_private_key. EVP_PKEY_keygen already generated an EVP_PKEY. - Add some missing error checks. Credit: David Benjamin - [Will Cosgrove brought this change] transport.c: socket is disconnected, return error (#500) File: transport.c Notes: This is to fix #102, instead of continuing to attempt to read a disconnected socket, it will now error out. Credit: TDi-jonesds - [Will Cosgrove brought this change] stale.yml Increasing stale values. Marc Hoersken (6 Jul 2020) - wincng: try newer DH API first, fallback to legacy RSA API Avoid the use of RtlGetVersion or similar Win32 functions, since these depend on version information from manifests. This commit makes the WinCNG backend first try to use the new DH algorithm API with the raw secret derivation feature. In case this feature is not available the WinCNG backend will fallback to the classic approach of using RSA-encrypt to perform the required modular exponentiation of BigNums. The feature availability test is done during the first handshake and the result is stored in the crypto backends global state. Follow up to #397 Closes #484 - wincng: fix indentation of function arguments and comments Follow up to #397 - [Wez Furlong brought this change] wincng: use newer DH API for Windows 8.1+ Since Windows 1903 the approach used to perform DH kex with the CNG API has been failing. This commit switches to using the `DH` algorithm provider to perform generation of the key pair and derivation of the shared secret. It uses a feature of CNG that is not yet documented. The sources of information that I've found on this are: * https://stackoverflow.com/a/56378698/149111 * https://github.com/wbenny/mini-tor/blob/5d39011e632be8e2b6b1819ee7295e8bd9b7a769/mini/crypto/cng/dh.inl#L355 With this change I am able to successfully connect from Windows 10 to my ubuntu system. Refs: https://github.com/alexcrichton/ssh2-rs/issues/122 Fixes: https://github.com/libssh2/libssh2/issues/388 Closes: https://github.com/libssh2/libssh2/pull/397 GitHub (1 Jul 2020) - [Zenju brought this change] comp.c: Fix name clash with ZLIB macro "compress" (#418) File: comp.c Notes: * Fix name clash with ZLIB macro "compress". Credit: Zenju - [yann-morin-1998 brought this change] buildsystem: drop custom buildconf script, rely on autoreconf (#224) Notes: The buildconf script is currently required, because we need to copy a header around, because it is used both from the library and the examples sources. However, having a custom 'buildconf'-like script is not needed if we can ensure that the header exists by the time it is needed. For that, we can just append the src/ directory to the headers search path for the examples. And then it means we no longer need to generate the same header twice, so we remove the second one from configure.ac. Now, we can just call "autoreconf -fi" to generate the autotools files, instead of relying on the canned sequence in "buildconf", since autoreconf has now long known what to do at the correct moment (future versions of autotools, automake, autopoint, autoheader etc... may require an other ordering, or other intermediate steps, etc...). Eventually, get rid of buildconf now it is no longer needed. In fact, we really keep it for legacy, but have it just call autoreconf (and print a nice user-friendly warning). Don't include it in the release tarballs, though. Update doc, gitignore, and travis-CI jobs accordingly. Credit: Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Sam Voss <sam.voss@rockwellcollins.com> - [Will Cosgrove brought this change] libssh2.h: Update Diffie Hellman group values (#493) File: libssh2.h Notes: Update the min, preferred and max DH group values based on RFC 8270. Credit: Will Cosgrove, noted from email list by Mitchell Holland Marc Hoersken (22 Jun 2020) - travis: use existing Makefile target to run checksrc - Makefile: also run checksrc on test source files - tests: avoid use of deprecated function _sleep (#490) - tests: avoid use of banned function strncat (#489) - tests: satisfy checksrc regarding max line length of 79 chars Follow up to 2764bc8e06d51876b6796d6080c6ac51e20f3332 - tests: satisfy checksrc with whitespace only fixes checksrc.pl -i4 -m79 -ASIZEOFNOPAREN -ASNPRINTF -ACOPYRIGHT -AFOPENMODE tests/*.[ch] - tests: add support for ports published via Docker for Windows - tests: restore retry behaviour for docker-machine ip command - tests: fix mix of declarations and code failing C89 compliance - wincng: add and improve checks in bit counting function - wincng: align bits to bytes calculation in all functions - wincng: do not disable key validation that can be enabled The modular exponentiation also works with key validation enabled. - wincng: fix return value in _libssh2_dh_secret Do not ignore return value of modular exponentiation. - appveyor: build and run tests for WinCNG crypto backend GitHub (1 Jun 2020) - [suryakalpo brought this change] INSTALL_CMAKE.md: Update formatting (#481) File: INSTALL_CMAKE.md Notes: Although the original text would be immediately clear to seasoned users of CMAKE and/or Unix shell, the lack of newlines may cause some confusion for newcomers. Hence, wrapping the texts in a md code-block such that the newlines appear as intended. credit: suryakalpo Marc Hoersken (31 May 2020) - src: add new and align include guards in header files (#480) Make sure all include guards exist and follow the same format. - wincng: fix multiple definition of `_libssh2_wincng' (#479) Add missing include guard and move global state from header to source file by using extern. GitHub (28 May 2020) - [Will Cosgrove brought this change] transport.c: moving total_num check from #476 (#478) file: transport.c notes: moving total_num zero length check from #476 up to the prior bounds check which already includes a total_num check. Makes it slightly more readable. credit: Will Cosgrove - [lutianxiong brought this change] transport.c: fix use-of-uninitialized-value (#476) file:transport.c notes: return error if malloc(0) credit: lutianxiong - [Dr. Koutheir Attouchi brought this change] libssh2_sftp.h: Changed type of LIBSSH2_FX_* constants to unsigned long, fixes #474 File: libssh2_sftp.h Notes: Error constants `LIBSSH2_FX_*` are only returned by `libssh2_sftp_last_error()` which returns `unsigned long`. Therefore these constants should be defined as unsigned long literals, instead of int literals. Credit: Dr. Koutheir Attouchi - [monnerat brought this change] os400qc3.c: constify libssh2_os400qc3_hash_update() data parameter. (#469) Files: os400qc3.c, os400qc3.h Notes: Fixes building on OS400. #426 Credit: Reported-by: hjindra on github, dev by Monnerat - [monnerat brought this change] HACKING.CRYPTO: keep up to date with new crypto definitions from code. (#466) File: HACKING.CRYPTO Notes: This commit updates the HACKING.CRYPTO documentation file in an attempt to make it in sync with current code. New documented features are: SHA384 SHA512 ECDSA ED25519 Credit: monnerat - [Harry Sintonen brought this change] kex.c: Add diffie-hellman-group14-sha256 Key Exchange Method (#464) File: kex.c Notes: Added diffie-hellman-group14-sha256 kex Credit: Harry Sintonen <sintonen@iki.fi> - [Will Cosgrove brought this change] os400qc3.h: define sha512 macros (#465) file: os400qc3.h notes: fixes for building libssh2 1.9.x - [Will Cosgrove brought this change] os400qc3.h: define EC types to fix building #426 (#462) File: os400qc3.h Notes: define missing EC types which prevents building Credit: hjindra - [Brendan Shanks brought this change] hostkey.c: Fix 'unsigned int'/'uint32_t' mismatch (#461) File: hostkey.c Notes: These types are the same size so most compilers are fine with it, but CodeWarrior (on classic MacOS) throws an ‘illegal implicit conversion’ error Credit: Brendan Shanks - [Thomas Klausner brought this change] Makefile.am: Fix unportable test(1) operator. (#459) file: Makefile.am Notes: The POSIX comparison operator for test(1) is =; bash supports == but not even test from GNU coreutils does. Credit: Thomas Klausner - [Tseng Jun brought this change] openssl.c: minor changes of coding style (#454) File: openssl.c Notes: minor changes of coding style and align preprocessor conditional for #439 Credit: Tseng Jun - [Hans Meier brought this change] openssl.c: Fix for use of uninitialized aes_ctr_cipher.key_len (#453) File: Openssl.c Notes: * Fix for use of uninitialized aes_ctr_cipher.key_len when using HAVE_OPAQUE_STRUCTS, regression from #439 Credit: Hans Meirer, Tseng Jun - [Zenju brought this change] agent.c: Fix Unicode builds on Windows (#417) File: agent.c Notes: Fixes unicode builds for Windows in Visual Studio 16.3.2. Credit: Zenju - [Hans Meier brought this change] openssl.c: Fix use-after-free crash in openssl backend without memory leak (#439) Files: openssl.c Notes: Fixes memory leaks and use after free AES EVP_CIPHER contexts when using OpenSSL 1.0.x. Credit: Hans Meier - [Romain Geissler @ Amadeus brought this change] Session.c: Fix undefined warning when mixing with LTO-enabled libcurl. (#449) File: Session.c Notes: With gcc 9, libssh2, libcurl and LTO enabled for all binaries I see this warning (error with -Werror): vssh/libssh2.c: In function ‘ssh_statemach_act’: /data/mwrep/rgeissler/ospack/ssh2/BUILD/libssh2-libssh2-03c7c4a/src/session.c:579:9: error: ‘seconds_to_next’ is used uninitialized in this function [-Werror=uninitialized] 579 | int seconds_to_next; | ^ lto1: all warnings being treated as errors Gcc normally issues -Wuninitialized when it is sure there is a problem, and -Wmaybe-uninitialized when it's not sure, but it's possible. Here the compiler seems to have find a real case where this could happen. I looked in your code and overall it seems you always check if the return code is non null, not often that it's below zero. I think we should do the same here. With this patch, gcc is fine. Credit: Romain-Geissler-1A - [Zenju brought this change] transport.c: Fix crash with delayed compression (#443) Files: transport.c Notes: Fixes crash with delayed compression option using Bitvise server. Contributor: Zenju - [Will Cosgrove brought this change] Update INSTALL_MAKE path to INSTALL_MAKE.md (#446) Included for #429 - [Will Cosgrove brought this change] Update INSTALL_CMAKE filename to INSTALL_CMAKE.md (#445) Fixing for #429 - [Wallace Souza brought this change] Rename INSTALL_CMAKE to INTALL_CMAKE.md (#429) Adding Markdown file extension in order to Github render the instructions properly Will Cosgrove (17 Dec 2019) - [Daniel Stenberg brought this change] include/libssh2.h: fix comment: the known host key uses 4 bits (#438) - [Zenju brought this change] ssh-ed25519: Support PKIX + calc pubkey from private (#416) Files: openssl.c/h Author: Zenju Notes: Adds support for PKIX key reading by fixing: _libssh2_pub_priv_keyfile() is missing the code to extract the ed25519 public key from a given private key _libssh2_ed25519_new_private_frommemory is only parsing the openssh key format but does not understand PKIX (as retrieved via PEM_read_bio_PrivateKey) GitHub (15 Oct 2019) - [Will Cosgrove brought this change] .travis.yml: Fix Chrome and 32 bit builds (#423) File: .travis.yml Notes: * Fix Chrome installing by using Travis build in directive * Update to use libgcrypt20-dev package to fix 32 bit builds based on comments found here: https://launchpad.net/ubuntu/xenial/i386/libgcrypt11-dev - [Will Cosgrove brought this change] packet.c: improved parsing in packet_x11_open (#410) Use new API to parse data in packet_x11_open() for better bounds checking. Will Cosgrove (12 Sep 2019) - [Michael Buckley brought this change] knownhost.c: Double the static buffer size when reading and writing known hosts (#409) Notes: We had a user who was being repeatedly prompted to accept a server key repeatedly. It turns out the base64-encoded key was larger than the static buffers allocated to read and write known hosts. I doubled the size of these buffers. Credit: Michael Buckley GitHub (4 Sep 2019) - [Will Cosgrove brought this change] packet.c: improved packet parsing in packet_queue_listener (#404) * improved bounds checking in packet_queue_listener file: packet.c notes: improved parsing packet in packet_queue_listener - [Will Cosgrove brought this change] packet.c: improve message parsing (#402) * packet.c: improve parsing of packets file: packet.c notes: Use _libssh2_get_string API in SSH_MSG_DEBUG/SSH_MSG_DISCONNECT. Additional uint32 bounds check in SSH_MSG_GLOBAL_REQUEST. - [Will Cosgrove brought this change] misc.c: _libssh2_ntohu32 cast bit shifting (#401) To quite overly aggressive analyzers. Note, the builds pass, Travis is having some issues with Docker images. - [Will Cosgrove brought this change] kex.c: improve bounds checking in kex_agree_methods() (#399) file: kex.c notes: use _libssh2_get_string instead of kex_string_pair which does additional checks Will Cosgrove (23 Aug 2019) - [Fabrice Fontaine brought this change] acinclude.m4: add mbedtls to LIBS (#371) Notes: This is useful for static builds so that the Libs.private field in libssh2.pc contains correct info for the benefit of pkg-config users. Static link with libssh2 requires this information. Signed-off-by: Baruch Siach <baruch@tkos.co.il> [Retrieved from: https://git.buildroot.net/buildroot/tree/package/libssh2/0002-acinclude.m4-add-mbedtls-to-LIBS.patch] Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com> Credit: Fabrice Fontaine - [jethrogb brought this change] Generate debug info when building with MSVC (#178) files: CMakeLists.txt notes: Generate debug info when building with MSVC credit: jethrogb - [Panos brought this change] Add agent forwarding implementation (#219) files: channel.c, test_agent_forward_succeeds.c, libssh2_priv.h, libssh2.h, ssh2_agent_forwarding.c notes: * Adding SSH agent forwarding. * Fix agent forwarding message, updated example. Added integration test code and cmake target. Added example to cmake list. credit: pkittenis GitHub (2 Aug 2019) - [Will Cosgrove brought this change] Update EditorConfig Added max_line_length = 80 - [Will Cosgrove brought this change] global.c : fixed call to libssh2_crypto_exit #394 (#396) * global.c : fixed call to libssh2_crypto_exit #394 File: global.c Notes: Don't call `libssh2_crypto_exit()` until `_libssh2_initialized` count is down to zero. Credit: seba30 Will Cosgrove (30 Jul 2019) - [hlefebvre brought this change] misc.c : Add an EWOULDBLOCK check for better portability (#172) File: misc.c Notes: Added support for all OS' that implement EWOULDBLOCK, not only VMS Credit: hlefebvre - [Etienne Samson brought this change] userauth.c: fix off by one error when loading public keys with no id (#386) File: userauth.c Credit: Etienne Samson Notes: Caught by ASAN: ================================================================= ==73797==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60700001bcf0 at pc 0x00010026198d bp 0x7ffeefbfed30 sp 0x7ffeefbfe4d8 READ of size 69 at 0x60700001bcf0 thread T0 2019-07-04 08:35:30.292502+0200 atos[73890:2639175] examining /Users/USER/*/libssh2_clar [73797] #0 0x10026198c in wrap_memchr (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x1f98c) #1 0x1000f8e66 in file_read_publickey userauth.c:633 #2 0x1000f2dc9 in userauth_publickey_fromfile userauth.c:1513 #3 0x1000f2948 in libssh2_userauth_publickey_fromfile_ex userauth.c:1590 #4 0x10000e254 in test_userauth_publickey__ed25519_auth_ok publickey.c:69 #5 0x1000090c3 in clar_run_test clar.c:260 #6 0x1000038f3 in clar_run_suite clar.c:343 #7 0x100003272 in clar_test_run clar.c:522 #8 0x10000c3cc in main runner.c:60 #9 0x7fff5b43b3d4 in start (libdyld.dylib:x86_64+0x163d4) 0x60700001bcf0 is located 0 bytes to the right of 80-byte region [0x60700001bca0,0x60700001bcf0) allocated by thread T0 here: #0 0x10029e053 in wrap_malloc (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x5c053) #1 0x1000b4978 in libssh2_default_alloc session.c:67 #2 0x1000f8aba in file_read_publickey userauth.c:597 #3 0x1000f2dc9 in userauth_publickey_fromfile userauth.c:1513 #4 0x1000f2948 in libssh2_userauth_publickey_fromfile_ex userauth.c:1590 #5 0x10000e254 in test_userauth_publickey__ed25519_auth_ok publickey.c:69 #6 0x1000090c3 in clar_run_test clar.c:260 #7 0x1000038f3 in clar_run_suite clar.c:343 #8 0x100003272 in clar_test_run clar.c:522 #9 0x10000c3cc in main runner.c:60 #10 0x7fff5b43b3d4 in start (libdyld.dylib:x86_64+0x163d4) SUMMARY: AddressSanitizer: heap-buffer-overflow (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x1f98c) in wrap_memchr Shadow bytes around the buggy address: 0x1c0e00003740: fd fd fd fd fd fd fd fd fd fd fa fa fa fa fd fd 0x1c0e00003750: fd fd fd fd fd fd fd fa fa fa fa fa 00 00 00 00 0x1c0e00003760: 00 00 00 00 00 00 fa fa fa fa 00 00 00 00 00 00 0x1c0e00003770: 00 00 00 fa fa fa fa fa fd fd fd fd fd fd fd fd 0x1c0e00003780: fd fd fa fa fa fa fd fd fd fd fd fd fd fd fd fa =>0x1c0e00003790: fa fa fa fa 00 00 00 00 00 00 00 00 00 00[fa]fa 0x1c0e000037a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x1c0e000037b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x1c0e000037c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x1c0e000037d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x1c0e000037e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb Shadow gap: cc - [Thilo Schulz brought this change] openssl.c : Fix use-after-free crash on reinitialization of openssl backend file : openssl.c notes : libssh2's openssl backend has a use-after-free condition if HAVE_OPAQUE_STRUCTS is defined and you call libssh2_init() again after prior initialisation/deinitialisation of libssh2 credit : Thilo Schulz - [axjowa brought this change] openssl.h : Use of ifdef where if should be used (#389) File : openssl.h Notes : LIBSSH2_ECDSA and LIBSSH2_ED25519 are always defined so the #ifdef checks would never be false. This change makes it possible to build libssh2 against OpenSSL built without EC support. Change-Id: I0a2f07c2d80178314dcb7d505d1295d19cf15afd Credit : axjowa - [Zenju brought this change] Agent.c : Preserve error info from agent_list_identities() (#374) Files : agent.c Notes : Currently the error details as returned by agent_transact_pageant() are overwritten by a generic "agent list id failed" message by int agent_list_identities(LIBSSH2_AGENT* agent). Credit : Zenju - [Who? Me?! brought this change] Channel.c: Make sure the error code is set in _libssh2_channel_open() (#381) File : Channel.c Notes : if _libssh2_channel_open() fails, set the error code. Credit : mark-i-m - [Orgad Shaneh brought this change] Kex.c, Remove unneeded call to strlen (#373) File : Kex.c Notes : Removed call to strlen Credit : Orgad Shaneh - [Pedro Monreal brought this change] Spelling corrections (#380) Files : libssh2.h, libssh2_sftp.h, bcrypt_pbkdf.c, mbedtls.c, sftp.c, ssh2.c Notes : * Fixed misspellings Credit : Pedro Monreal - [Sebastián Katzer brought this change] Fix Potential typecast error for `_libssh2_ecdsa_key_get_curve_type` (#383) Issue : #383 Files : hostkey.c, crypto.h, openssl.c Notes : * Fix potential typecast error for `_libssh2_ecdsa_key_get_curve_type` * Rename _libssh2_ecdsa_key_get_curve_type to _libssh2_ecdsa_get_curve_type Credit : Sebastián Katzer GitHub (20 Jun 2019) - [Will Cosgrove brought this change] bump copyright date Version 1.9.0 (19 Jun 2019) GitHub (19 Jun 2019) - [Will Cosgrove brought this change] 1.9 Formatting - [Will Cosgrove brought this change] 1.9 Release notes Will Cosgrove (17 May 2019) - [Alexander Curtiss brought this change] libgcrypt.c : Fixed _libssh2_rsa_sha1_sign memory leak. (#370) File: libgcrypt.c Notes : Added calls to gcry_sexp_release to free memory allocated by gcry_sexp_find_token Credit : Reporter : beckmi PR by: Alexander Curtiss - [Orivej Desh brought this change] libssh2_priv.h : Fix musl build warning on sys/poll.h (#346) File : libssh2_priv.h Notes : musl prints `redirecting incorrect #include <sys/poll.h> to <poll.h>` http://git.musl-libc.org/cgit/musl/commit/include/sys/poll.h?id=54446d730cfb17c5f7bcf57f139458678f5066cc poll is defined by POSIX to be in poll.h: http://pubs.opengroup.org/onlinepubs/7908799/xsh/poll.html Credit : Orivej Desh GitHub (1 May 2019) - [Will Cosgrove brought this change] kex.c : additional bounds checks in diffie_hellman_sha1/256 (#361) Files : kex.c, misc.c, misc.h Notes : Fixed possible out of bounds memory access when reading malformed data in diffie_hellman_sha1() and diffie_hellman_sha256(). Added _libssh2_copy_string() to misc.c to return an allocated and filled char buffer from a string_buf offset. Removed no longer needed s var in kmdhgGPshakex_state_t. Will Cosgrove (26 Apr 2019) - [Tseng Jun brought this change] sftp.c : sftp_bin2attr() Correct attrs->gid assignment (#366) Regression with fix for #339 Credit : Tseng Jun - [Tseng Jun brought this change] kex.c : Correct type cast in curve25519_sha256() (#365) GitHub (24 Apr 2019) - [Will Cosgrove brought this change] transport.c : scope local total_num var (#364) file : transport.c notes : move local `total_num` variable inside of if block to prevent scope access issues which caused #360. Will Cosgrove (24 Apr 2019) - [doublex brought this change] transport.c : fixes bounds check if partial packet is read Files : transport.c Issue : #360 Notes : 'p->total_num' instead of local value total_num when doing bounds check. Credit : Doublex GitHub (23 Apr 2019) - [Will Cosgrove brought this change] Editor config file for source files (#322) Simple start to an editor config file when editing source files to make sure they are configured correctly. - [Will Cosgrove brought this change] misc.c : String buffer API improvements (#332) Files : misc.c, hostkey.c, kex.c, misc.h, openssl.c, sftp.c Notes : * updated _libssh2_get_bignum_bytes and _libssh2_get_string. Now pass in length as an argument instead of returning it to keep signedness correct. Now returns -1 for failure, 0 for success. _libssh2_check_length now returns 0 on success and -1 on failure to match the other string_buf functions. Added comment to _libssh2_check_length. Credit : Will Cosgrove Will Cosgrove (19 Apr 2019) - [doublex brought this change] mbedtls.c : _libssh2_mbedtls_rsa_new_private_frommemory() allow private-key from memory (#359) File : mbedtls.c Notes: _libssh2_mbedtls_rsa_new_private_frommemory() fixes private-key from memory reading to by adding NULL terminator before parsing; adds passphrase support. Credit: doublex - [Ryan Kelley brought this change] Session.c : banner_receive() from leaking when accessing non ssh ports (#356) File : session.c Release previous banner in banner_receive() if the session is reused after a failed connection. Credit : Ryan Kelley GitHub (11 Apr 2019) - [Will Cosgrove brought this change] Formatting in agent.c Removed whitespace. - [Will Cosgrove brought this change] Fixed formatting in agent.c Quiet linter around a couple if blocks and pointer. Will Cosgrove (11 Apr 2019) - [Zhen-Huan HWANG brought this change] sftp.c : discard and reset oversized packet in sftp_packet_read() (#269) file : sftp.c notes : when sftp_packet_read() encounters an sftp packet which exceeds SFTP max packet size it now resets the reading state so it can continue reading. credit : Zhen-Huan HWANG GitHub (11 Apr 2019) - [Will Cosgrove brought this change] Add agent functions libssh2_agent_get_identity_path() and libssh2_agent_set_identity_path() (#308) File : agent.c Notes : Libssh2 uses the SSH_AUTH_SOCK env variable to read the system agent location. However, when using a custom agent path you have to set this value using setenv which is not thread-safe. The new functions allow for a way to set a custom agent socket path in a thread safe manor. - [Will Cosgrove brought this change] Simplified _libssh2_check_length (#350) * Simplified _libssh2_check_length misc.c : _libssh2_check_length() Removed cast and improved bounds checking and format. Credit : Yuriy M. Kaminskiy - [Will Cosgrove brought this change] _libssh2_check_length() : additional bounds check (#348) Misc.c : _libssh2_check_length() Ensure the requested length is less than the total length before doing the additional bounds check Daniel Stenberg (25 Mar 2019) - misc: remove 'offset' from string_buf It isn't necessary. Closes #343 - sftp: repair mtime from e1ead35e475 A regression from e1ead35e4759 broke the SFTP mtime logic in sftp_bin2attr Also simplified the _libssh2_get_u32/u64 functions slightly. Closes #342 - session_disconnect: don't zero state, just clear the right bit If we clear the entire field, the freeing of data in session_free() is skipped. Instead just clear the bit that risk making the code get stuck in the transport functions. Regression from 4d66f6762ca3fc45d9. Reported-by: dimmaq on github Fixes #338 Closes #340 - libssh2_sftp.h: restore broken ABI Commit 41fbd44 changed variable sizes/types in a public struct which broke the ABI, which breaks applications! This reverts that change. Closes #339 - style: make includes and examples code style strict make travis and the makefile rule verify them too Closes #334 GitHub (21 Mar 2019) - [Daniel Stenberg brought this change] create a github issue template Daniel Stenberg (21 Mar 2019) - stale-bot: activated The stale bot will automatically mark stale issues (inactive for 90 days) and if still untouched after 21 more days, close them. See https://probot.github.io/apps/stale/ - libssh2_session_supported_algs.3: fix formatting mistakes Reported-by: Max Horn Fixes #57 - [Zenju brought this change] libssh2.h: Fix Error C2371 'ssize_t': redefinition Closes #331 - travis: add code style check Closes #324 - code style: unify code style Indent-level: 4 Max columns: 79 No spaces after if/for/while Unified brace positions Unified white spaces - src/checksrc.pl: code style checker imported as-is from curl Will Cosgrove (19 Mar 2019) - Merge branch 'MichaelBuckley-michaelbuckley-security-fixes' - Silence unused var warnings (#329) Silence warnings about unused variables in this test - Removed unneeded > 0 check When checking `userauth_kybd_num_prompts > 100` we don't care if it's also above zero. - [Matthew D. Fuller brought this change] Spell OpenSS_H_ right when talking about their specific private key (#321) Good catch, thanks. GitHub (19 Mar 2019) - [Will Cosgrove brought this change] Silence unused var warnings (#329) Silence warnings about unused variables in this test Michael Buckley (19 Mar 2019) - Fix more scope and printf warning errors - Silence unused variable warning GitHub (19 Mar 2019) - [Will Cosgrove brought this change] Removed unneeded > 0 check When checking `userauth_kybd_num_prompts > 100` we don't care if it's also above zero. Will Cosgrove (19 Mar 2019) - [Matthew D. Fuller brought this change] Spell OpenSS_H_ right when talking about their specific private key (#321) Good catch, thanks. Michael Buckley (18 Mar 2019) - Fix errors identified by the build process - Fix casting errors after merge GitHub (18 Mar 2019) - [Michael Buckley brought this change] Merge branch 'master' into michaelbuckley-security-fixes Michael Buckley (18 Mar 2019) - Move fallback SIZE_MAX and UINT_MAX to libssh2_priv.h - Fix type and logic issues with _libssh2_get_u64 Daniel Stenberg (17 Mar 2019) - examples: fix various compiler warnings - lib: fix various compiler warnings - session: ignore pedantic warnings for funcpointer <=> void * - travis: add a build using configure Closes #320 - configure: provide --enable-werror - appveyor: remove old builds that mostly cause failures ... and only run on master branch. Closes #323 - cmake: add two missing man pages to get installed too Both libssh2_session_handshake.3 and libssh2_userauth_publickey_frommemory.3 were installed by the configure build already. Reported-by: Arfrever on github Fixes #278 - include/libssh2.h: warning: "_WIN64" is not defined, evaluates to 0 We don't use #if for defines that might not be defined. - pem: //-comments are not allowed Will Cosgrove (14 Mar 2019) - [Daniel Stenberg brought this change] userauth: fix "Function call argument is an uninitialized value" (#318) Detected by scan-build. - fixed unsigned/signed issue Daniel Stenberg (15 Mar 2019) - session_disconnect: clear state If authentication is started but not completed before the application gives up and instead wants to shut down the session, the '->state' field might still be set and thus effectively dead-lock session_disconnect. This happens because both _libssh2_transport_send() and _libssh2_transport_read() refuse to do anything as long as state is set without the LIBSSH2_STATE_KEX_ACTIVE bit. Reported in curl bug https://github.com/curl/curl/issues/3650 Closes #310 Will Cosgrove (14 Mar 2019) - Release notes from 1.8.1 Michael Buckley (14 Mar 2019) - Use string_buf in sftp_init(). - Guard against out-of-bounds reads in publickey.c - Guard against out-of-bounds reads in session.c - Guard against out-of-bounds reads in userauth.c - Use LIBSSH2_ERROR_BUFFER_TOO_SMALL instead of LIBSSH2_ERROR_OUT_OF_BOUNDARY in sftp.c - Additional bounds checking in sftp.c - Additional length checks to prevent out-of-bounds reads and writes in _libssh2_packet_add(). https://libssh2.org/CVE-2019-3862.html - Add a required_size parameter to sftp_packet_require et. al. to require callers of these functions to handle packets that are too short. https://libssh2.org/CVE-2019-3860.html - Check the length of data passed to sftp_packet_add() to prevent out-of-bounds reads. - Prevent zero-byte allocation in sftp_packet_read() which could lead to an out-of-bounds read. https://libssh2.org/CVE-2019-3858.html - Sanitize padding_length - _libssh2_transport_read(). https://libssh2.org/CVE-2019-3861.html This prevents an underflow resulting in a potential out-of-bounds read if a server sends a too-large padding_length, possibly with malicious intent. - Defend against writing beyond the end of the payload in _libssh2_transport_read(). - Defend against possible integer overflows in comp_method_zlib_decomp. GitHub (14 Mar 2019) - [Will Cosgrove brought this change] Security fixes (#315) * Bounds checks Fixes for CVEs https://www.libssh2.org/CVE-2019-3863.html https://www.libssh2.org/CVE-2019-3856.html * Packet length bounds check CVE https://www.libssh2.org/CVE-2019-3855.html * Response length check CVE https://www.libssh2.org/CVE-2019-3859.html * Bounds check CVE https://www.libssh2.org/CVE-2019-3857.html * Bounds checking CVE https://www.libssh2.org/CVE-2019-3859.html and additional data validation * Check bounds before reading into buffers * Bounds checking CVE https://www.libssh2.org/CVE-2019-3859.html * declare SIZE_MAX and UINT_MAX if needed - [Will Cosgrove brought this change] fixed type warnings (#309) - [Will Cosgrove brought this change] Bumping version number for pending 1.8.1 release Will Cosgrove (4 Mar 2019) - [Daniel Stenberg brought this change] _libssh2_string_buf_free: use correct free (#304) Use LIBSSH2_FREE() here, not free(). We allow memory function replacements so free() is rarely the right choice... GitHub (26 Feb 2019) - [Will Cosgrove brought this change] Fix for building against libreSSL #302 Changed to use the check we use elsewhere. - [Will Cosgrove brought this change] Fix for when building against LibreSSL #302 Will Cosgrove (25 Feb 2019) - [gartens brought this change] docs: update libssh2_hostkey_hash.3 [ci skip] (#301) GitHub (21 Feb 2019) - [Will Cosgrove brought this change] fix malloc/free mismatches #296 (#297) - [Will Cosgrove brought this change] Replaced malloc with calloc #295 - [Will Cosgrove brought this change] Abstracted OpenSSL calls out of hostkey.c (#294) - [Will Cosgrove brought this change] Fix memory dealloc impedance mis-match #292 (#293) When using ed25519 host keys and a custom memory allocator. - [Will Cosgrove brought this change] Added call to OpenSSL_add_all_digests() #288 For OpenSSL 1.0.x we need to call OpenSSL_add_all_digests(). Will Cosgrove (12 Feb 2019) - [Zhen-Huan HWANG brought this change] SFTP: increase maximum packet size to 256K (#268) to match implementations like OpenSSH. - [Zenju brought this change] Fix https://github.com/libssh2/libssh2/pull/271 (#284) GitHub (16 Jan 2019) - [Will Cosgrove brought this change] Agent NULL check in shutdown #281 Will Cosgrove (15 Jan 2019) - [Adrian Moran brought this change] mbedtls: Fix leak of 12 bytes by each key exchange. (#280) Correctly free ducts by calling _libssh2_mbedtls_bignum_free() in dtor. - [alex-weaver brought this change] Fix error compiling on Win32 with STDCALL=ON (#275) GitHub (8 Nov 2018) - [Will Cosgrove brought this change] Allow default permissions to be used in sftp_mkdir (#271) Added constant LIBSSH2_SFTP_DEFAULT_MODE to use the server default permissions when making a new directory Will Cosgrove (13 Sep 2018) - [Giulio Benetti brought this change] openssl: fix dereferencing ambiguity potentially causing build failure (#267) When dereferencing from *aes_ctr_cipher, being a pointer itself, ambiguity can occur; fixed possible build errors. Viktor Szakats (12 Sep 2018) - win32/GNUmakefile: define HAVE_WINDOWS_H This macro was only used in test/example code before, now it is also used in library code, but only defined automatically by automake/cmake, so let's do the same for the standalone win32 make file. It'd be probably better to just rely on the built-in _WIN32 macro to detect the presence of windows.h though. It's already used in most of libssh2 library code. There is a 3rd, similar macro named LIBSSH2_WIN32, which might also be replaced with _WIN32. Ref: https://github.com/libssh2/libssh2/commit/8b870ad771cbd9cd29edbb3dbb0878e950f868ab Closes https://github.com/libssh2/libssh2/pull/266 Marc Hoersken (2 Sep 2018) - Fix conditional check for HAVE_DECL_SECUREZEROMEMORY "Unlike the other `AC_CHECK_*S' macros, when a symbol is not declared, HAVE_DECL_symbol is defined to `0' instead of leaving HAVE_DECL_symbol undeclared. When you are sure that the check was performed, use HAVE_DECL_symbol in #if." Source: autoconf documentation for AC_CHECK_DECLS. - Fix implicit declaration of function 'SecureZeroMemory' Include window.h in order to use SecureZeroMemory on Windows. - Fix implicit declaration of function 'free' by including stdlib.h GitHub (27 Aug 2018) - [Will Cosgrove brought this change] Use malloc abstraction function in pem parse Fix warning on WinCNG build. - [Will Cosgrove brought this change] Fixed possible junk memory read in sftp_stat #258 - [Will Cosgrove brought this change] removed INT64_C define (#260) No longer used. - [Will Cosgrove brought this change] Added conditional around engine.h include Will Cosgrove (6 Aug 2018) - [Alex Crichton brought this change] Fix OpenSSL link error with `no-engine` support (#259) This commit fixes linking against an OpenSSL library that was compiled with `no-engine` support by bypassing the initialization routines as they won't be available anyway. GitHub (2 Aug 2018) - [Will Cosgrove brought this change] ED25519 Key Support #39 (#248) OpenSSH Key and ED25519 support #39 Added _libssh2_explicit_zero() to explicitly zero sensitive data in memory #120 * ED25519 Key file support - Requires OpenSSL 1.1.1 or later * OpenSSH Key format reading support - Supports RSA/DSA/ECDSA/ED25519 types * New string buffer reading functions - These add build-in bounds checking and convenance methods. Used for OpenSSL PEM file reading. * Added new tests for OpenSSH formatted Keys - [Will Cosgrove brought this change] ECDSA key types are now explicit (#251) * ECDSA key types are now explicit Issue was brough up in pull request #248 Will Cosgrove (2 May 2018) - [Jakob Egger brought this change] Add Instructions for building from Master (#249) GitHub (27 Apr 2018) - [Will Cosgrove brought this change] Initialize sb_intl #226 Will Cosgrove (19 Apr 2018) - [doublex brought this change] buffer overflow (valgrind) (#159) - [Brendan Shanks brought this change] mbedTLS: Remove some C99-style intermingled variable declarations (#196) GitHub (18 Apr 2018) - [Will Cosgrove brought this change] fix for #160 Will Cosgrove (18 Apr 2018) - [doublex brought this change] fix memory leak when using mbedtls backend (#158) _libssh2_bn_init_from_bin/_libssh2_bn_free would leak bignum from mbedtls_calloc(). - [Brendan Shanks brought this change] mbedTLS: Avoid multiple definition errors for context handles (#197) - [Tseng Jun brought this change] Fix the EVP cipher meth memory leakage problem (#244) * Fix the EVP cipher meth memory leakage problem Looks good, thanks for the fixes. Marc Hörsken (31 Mar 2018) - [Will Cosgrove brought this change] Added ECDSA defines for WinCNG (#245) Fixed missing defines preventing building using WinCNG GitHub (30 Mar 2018) - [Will Cosgrove brought this change] Fix for _libssh2_rsa_new with OpenSSL 1.0.x missing d value assignment. Will Cosgrove (20 Mar 2018) - [Etienne Samson brought this change] A collection of small fixes (#198) * tests: Remove if-pyramids * tests: Switch run_command arguments * tests: Make run_command a vararg function * tests: Xcode doesn't obey CMake's test working directory * openssl: move manual AES-CTR cipher into crypto init * cmake: Move our include dir before all other include paths GitHub (15 Mar 2018) - [Will Cosgrove brought this change] Fixes incorrect indexing of KEX prefs string After stripping out an invalid KEX pref entry, it would incorrectly advance again leaving invalid values in the list. Viktor Szakats (13 Mar 2018) - tests: fix checksrc warnings Also: * add 'static' qualifier to file-wide const buffers * fix a non-ANSI C89 comment * silence a mismatched fprintf() mask warning by adding a cast - cmake: recognize OpenSSL 1.1 .dll names Also fix some comment typos and a stray tab. - docs: update an URL [ci skip] Daniel Stenberg (12 Mar 2018) - docs/SECURITY: the max embargo is 14 days now Viktor Szakats (12 Mar 2018) - docs: spelling fixes [ci skip] Closes https://github.com/libssh2/libssh2/pull/222 GitHub (12 Mar 2018) - [Will Cosgrove brought this change] Fixed minor tabs/spacing issues - [Will Cosgrove brought this change] Update kex.c - [Will Cosgrove brought this change] Added basic bounds checking #206 Basic bounds checking in ecdh_sha2_nistp() - [Will Cosgrove brought this change] Fixed Clang warning #206 Fixed possible garbage value for secret in an error case - [Will Cosgrove brought this change] Fixed incorrect #if to #ifdef #206 When checking HAVE_OPAQUE_STRUCTS. Viktor Szakats (12 Mar 2018) - src: suppress two checksrc warnings Ref: https://github.com/libssh2/libssh2/pull/235 - src: address fopen() warnings, add missing copyright headers Ref: https://github.com/libssh2/libssh2/pull/235 - src: replace sprintf() with snprintf() Ref: https://github.com/libssh2/libssh2/pull/235 - src: fix checksrc warnings Use checksrc.pl from the curl project, with (for now) suppressed long line warnings and indentation set to 4 spaces. Fixes are whitespace for the most part. Warning count went down from 2704 to 12. Also fix codespell typos, two non-ANSI C89 comments and a stray tab in include/libssh2.h. Ref: https://github.com/libssh2/libssh2/pull/235 - checksrc: add source style checker This is a slightly extended version of this original source from the curl project: https://github.com/curl/curl/blob/8b754c430b9a4c51aa606c687ee5014faf7c7b06/lib/checksrc.pl This version adds the following options to customize it for libssh2 (plus some whitespace formatting): `-i<n>` to override indentation spaces (2) `-m<n>` to override maximum line length (79) Command-line used to check libssh2 sources: $ ./checksrc.pl -i4 -m500 *.c *.h Closes https://github.com/libssh2/libssh2/pull/236 - src: add static qualifier To private, const strings. Closes https://github.com/libssh2/libssh2/pull/237 - [Will Cosgrove brought this change] Add support for ECDSA keys and host keys (#41) This commit lands full ECDSA key support when using the OpenSSL backend. Which includes: New KEX methods: ecdsa-sha2-nistp256, ecdsa-sha2-nistp384, ecdsa-sha2-nistp521 Can now read OpenSSL formatted ECDSA key files. Now supports known host keys of type ecdsa-sha2-nistp256. New curve types: NID_X9_62_prime256v1, NID_secp384r1, NID_secp521r1 Default host key preferred ordering is now nistp256, nistp384, nistp521, rsa, dss. Ref: https://github.com/libssh2/libssh2/issues/41 Closes https://github.com/libssh2/libssh2/pull/206 GitHub (15 Dec 2017) - [Will Cosgrove brought this change] Fixed possible crash when decoding invalid data When trying to decode invalid data, it frees the buffer but doesn't nil it so the caller gets a junk memory pointer which they could potentially double free. - [Will Cosgrove brought this change] Remove call to OpenSSL_add_all_ciphers() Now lives in libssh2 init() from PR #189 - [Will Cosgrove brought this change] Fixed incorrect reference to decrypted block Fixed incorrectly copied memory from p->buf into init instead of from the decrypted buffer block. The only reason this worked was because the crypt() function decrypts the value in-place and overwrites p->buf. I'm working on a fork that no longer does this and exposed this bug. Will Cosgrove (20 Oct 2017) - [Pan brought this change] Fix typo in crypt.c (#218) Kamil Dudka (17 Oct 2017) - session: avoid printing misleading debug messages ... while throwing LIBSSH2_ERROR_EAGAIN out of session_startup() If the session runs in blocking mode, LIBSSH2_ERROR_EAGAIN never reaches the libssh2 API boundary and, in non-blocking mode, these messages are suppressed by the condition in _libssh2_error_flags() anyway. Closes #211 Viktor Szakats (15 Oct 2017) - win32/GNUmakefile: allow customizing dll suffixes - New `LIBSSH2_DLL_SUFFIX` envvar will add a suffix to the generated libssh2 dll name. Useful to add `-x64` to 64-bit builds so that it can live in the same directory as the 32-bit one. By default this is empty. - New `LIBSSH2_DLL_A_SUFFIX` envvar to customize the suffix of the generated import library (implib) for libssh2 .dll. It defaults to `dll`, and it's useful to modify that to `.dll` to have the standard naming scheme for mingw-built .dlls, i.e. `libssh2.dll.a`. Ref: https://github.com/curl/curl/commit/aaa16f80256abc1463fd9374815130a165222257 Closes https://github.com/libssh2/libssh2/pull/215 - makefile.m32: allow to override gcc, ar and ranlib Allow to ovverride certain build tools, making it possible to use LLVM/Clang to build libssh2. The default behavior is unchanged. To build with clang (as offered by MSYS2), these settings can be used: LIBSSH2_CC=clang LIBSSH2_AR=llvm-ar LIBSSH2_RANLIB=llvm-ranlib Also adjust ranlib parameters to be compatible with LLVM/Clang's ranlib tool. Closes https://github.com/libssh2/libssh2/pull/214 GitHub (27 Sep 2017) - [Will Cosgrove brought this change] Fixes out of bounds memory access (#210) If an invalid PEM file is read and the lines are longer than 128 characters it will go out of bounds and crash on line 91. Will Cosgrove (11 Sep 2017) - [Kamil Dudka brought this change] scp: do not NUL-terminate the command for remote exec (#208) It breaks SCP download/upload from/to certain server implementations. The bug does not manifest with OpenSSH, which silently drops the NUL byte (eventually with any garbage that follows the NUL byte) before executing it. Bug: https://bugzilla.redhat.com/1489736 GitHub (21 Aug 2017) - [Viktor Szakats brought this change] openssl.c: remove no longer used variable (#204) after e378d2e30a40bd9bcee06dc3a4250f269098e200 - [Will Cosgrove brought this change] Fix for #188 (#189) * Update openssl.c * Create openssl.h Will Cosgrove (24 May 2017) - [Marcel Raad brought this change] openssl: fix build with OpenSSL 1.1 API (#176) When building with OPENSSL_API_COMPAT=0x10100000L, OpenSSL_add_all_algorithms and OpenSSL_add_all_ciphers don't exist. The corresponding functionality is handled automatically with OpenSSL 1.1. - [Sune Bredahl brought this change] Add support for SHA256 hostkey fingerprints (#180) Looks good, thanks! GitHub (12 May 2017) - [Will Cosgrove brought this change] Fix memory leak of crypt_ctx->h using openSSL 1.1+ (#177) Need to use EVP_CIPHER_CTX_free instead of EVP_CIPHER_CTX_reset. Marc Hoersken (2 Mar 2017) - tests/openssh_server/authorized_keys: add key_rsa_encrypted.pub - tests: add simple test for passphrase-protected PEM file support - os400qc3: enable passphrase-protected PEM file support using pem.c - pem: fix indentation and replace assert after 386e012292 - [Keno Fischer brought this change] pem: add passphrase-protected PEM file support for libgcrypt and wincng Since they use our own PEM parser which did not support encrypted PEM files, trying to use such files on these backends failed. Fix that by augmenting the PEM parser to support encrypted PEM files. - [Thomas brought this change] misc: use time constant implementation for AES CTR increment - [Thomas brought this change] wincng: add AES CTR mode support (aes128-ctr, aes192-ctr, aes256-ctr) - [Thomas brought this change] openssl: move shared AES-CTR code into misc Daniel Stenberg (20 Dec 2016) - [Alex Crichton brought this change] kex: acknowledge error code from libssh2_dh_key_pair() Fixes a segfault using ssh-agent on Windows This commit fixes a segfault seen dereferencing a null pointer on Windows when using ssh-agent. The problem ended up being that errors weren't being communicated all the way through, causing null pointers to be used when functions should have bailed out sooner. The `_libssh2_dh_key_pair` function for WinCNG was modified to propagate errors, and then the two callsites in kex.c of `diffie_hellman_sha{1,256}` were updated to recognize this error and bail out. Fixes #162 Closes #163 Alexander Lamaison (27 Nov 2016) - [monnerat brought this change] Implement Diffie-Hellman computations in crypto backends. (#149) Not all backends feature the low level API needed to compute a Diffie-Hellman secret, but some of them directly implement Diffie-Hellman support with opaque private data. The later approach is now generalized and backends are responsible for all Diffie Hellman computations. As a side effect, procedures/macros _libssh2_bn_rand and _libssh2_bn_mod_exp are no longer needed outside the backends. Peter Stuge (16 Nov 2016) - acinclude.m4: The mbedtls crypto backend actually requires libmbedcrypto Examples can't be linked with libmbedtls but need libmbedcrypto, and any users of libssh2 which use libtool and libssh2.la would encounter the same problem. This changes the mbedtls detection to search for libmbedcrypto, which is the actual dependency for the backend. - acinclude.m4: Add CPPFLAGS=-I$prefix-dir/include in LIBSSH2_LIB_HAVE_LINKFLAGS This is absolutely neccessary for header files to be found when AC_LIB_HAVE_LINKFLAGS searches for libraries. - acinclude.m4: Make saved variables in LIBSSH2_LIB_HAVE_LINKFLAGS uniform - docs/HACKING.CRYPTO: Improve documentation for autoconf build system Alexander Lamaison (16 Nov 2016) - [Alex Arslan brought this change] Check for netinet/in.h in the tests cmake file (#148) - [Patrick Monnerat brought this change] Define new Diffie-Hellman context for mbedTLS - [monnerat brought this change] Make libssh2 work again on os400. (#118) * os400: minimum supported OS version is now V6R1. Do not log compiler informational messages. * Implement crypto backend specific Diffie-Hellman computation. This feature is now needed on os400 because the QC3 library does not implement bn_mod_exp() natively. Up to now, this function was emulated using an RSA encryption, but commits ca5222ea819cc5ed797860070b4c6c1aeeb28420 and 7934c9ce2a029c43e3642a492d3b9e494d1542be (CVE-2016-0787) broke the emulation because QC3 only supports RSA exponents up to 512 bits. Happily, QC3 supports a native API for Diffie-Hellman computation, with opaque random value: this commit implements the use of this API and, as a side effect, enables support of this feature for any other crypto backend that would use it. A "generic" Diffie-Hellman computation internal API supports crypto backends not implementing their own: this generic API uses the same functions as before. * Fix typos in docs/HACKING.CRYPTO. - [Peter Stuge brought this change] acinclude.m4: Fixup OpenSSL EVP_aes_128_ctr() detection - [Peter Stuge brought this change] configure.ac: Add --with-crypto= instead of many different --with-$backend The new --with-crypto option replaces the previous backend-specific --with-{openssl,libgcrypt,mbedtls,wincng} options and fixes some issues. * libgcrypt or mbedtls would previously be used whenever found, even if configure was passed --without-libgcrypt or --without-mbedtls. * If --with-$backend was specified then configure would not fail even if that library could not be found, and would instead use whichever crypto library was found first. The new option defaults to `auto`, which makes configure check for all supported crypto libraries in turn, choosing the first one found, or exiting with an error if none can be found. - [Tony Kelman brought this change] Build mbedtls from source on Travis (#133) * Revert "Revert "travis: Test mbedtls too"" This reverts commit c4c60eac5ca756333034b07dd9e0b97741493ed3. * travis: Build mbedtls from source on Travis Use TOOLCHAIN_OPTION when calling cmake on mbedtls * tests: only run DSA tests for non-mbedtls crypto backends - [Peter Stuge brought this change] configure.ac src/Makefile.am: Remove dead AM_CONDITIONAL(OS400QC3) According to os400/README400 this backend can not be built with configure+make, and the conditional is hard coded to false. - [Peter Stuge brought this change] configure.ac: Add -DNDEBUG to CPPFLAGS in non-debug builds There are a few uses of assert() in channel.c, sftp.c and transport.c. - [Peter Stuge brought this change] src/global.c: Fix conditional AES-CTR support Most of libssh2 already has conditional support for AES-CTR according to the LIBSSH2_AES_CTR crypto backend #define, but global.c needed fixing. - [Peter Stuge brought this change] src/crypto.h src/userauth.c: Fix conditional RSA support Most of libssh2 already has conditional support for RSA according to the LIBSSH2_RSA crypto backend #define, but crypto.h and userauth.c needed a few small fixes. - [Peter Stuge brought this change] src/kex.c: Cast libssh2_sha{1,256}_update data arguments properly The update functions take a const unsigned char * but were called with (const) char * in some places, causing unneccessary warnings. - [Peter Stuge brought this change] docs/HACKING.CRYPTO: Fix two type typos - [Sergei Trofimovich brought this change] acinclude.m4: fix ./configure --with-libgcrypt The change fixes passing of bogus gcrypt prefix. Reproducible as: $ ./configure --with-libgcrypt $ make V=1 ... /bin/sh ../libtool --tag=CC --mode=link gcc -g -O2 -Iyes/include -version-info 1:1:0 -no-undefined -export-symbols-regex '^libssh2_.*' -lgcrypt -lz -Lyes/lib -o libssh2.la -rpath /usr/local/lib channel.lo comp.lo crypt.lo hostkey.lo kex.lo mac.lo misc.lo packet.lo publickey.lo scp.lo session.lo sftp.lo userauth.lo transport.lo version.lo knownhost.lo agent.lo libgcrypt.lo pem.lo keepalive.lo global.lo -lgcrypt ../libtool: line 7475: cd: yes/lib: No such file or directory libtool: error: cannot determine absolute directory name of 'yes/lib' These -Iyes/include -Lyes/lib come from libgcrypt code autodetection: if test -n "$use_libgcrypt" && test "$use_libgcrypt" != "no"; then LDFLAGS="$LDFLAGS -L$use_libgcrypt/lib" CFLAGS="$CFLAGS -I$use_libgcrypt/include" I assume it's a typo to use yes/no flag as a prefix and changed it to '$with_libgcrypt_prefix'. Reported-by: Mikhail Pukhlikov <cynede@gentoo.org> Signed-off-by: Sergei Trofimovich <siarheit@google.com> - [Zenju brought this change] libssh2_sftp_init hang: last error not set The problem is that the original if statement simply returns NULL, but does not set the session last error code. The consequence is that libssh2_sftp_init() also returns NULL and libssh2_session_last_errno(sshSession) == LIBSSH2_ERROR_NONE. In my test the LIBSSH2_ERROR_EAGAIN is coming from sftp.c row 337: if(4 != sftp->partial_size_len) /* we got a short read for the length part */ return LIBSSH2_ERROR_EAGAIN; with "partial_size_len == 0". Not sure if this is expected. - [Aidan Hobson Sayers brought this change] docs: correctly describe channel_wait_eof channel_wait_eof waits for channel->remote.eof, which is set on receiving a `SSH_MSG_CHANNEL_EOF` message. This message is sent when a party has no more data to send on a channel. - [Zenju brought this change] Fix MSVC 14 compilation warning (#92) 1> sftp.c 1>libssh2-files\src\sftp.c(3393): warning C4456: declaration of 'retcode' hides previous local declaration 1> libssh2-files\src\sftp.c(3315): note: see declaration of 'retcode' - [Salvador Fandino brought this change] LIBSSH2_ERROR_CHANNEL_WINDOW_FULL: add new error code In order to signal that the requested operation can not succeed because the receiving window had been exhausted, the error code LIBSSH2_ERROR_BUFFER_TOO_SMALL has been reused but I have found that in certain context it may be ambigous. This patch introduces a new error code, LIBSSH2_ERROR_CHANNEL_WINDOW_FULL, exclusive to signal that condition. - [Salvador Fandino brought this change] channel_wait_eof: handle receive window exhaustion Until now, in blocking mode, if the remote receiving window is exhausted this function hangs forever as data is not read and the remote side just keeps waiting for the window to grow before sending more data. This patch, makes this function check for that condition and abort with an error when it happens. - [Salvador Fandino brought this change] channel_wait_closed: don't fail when unread data is queued This function was calling channel_wait_eof to ensure that the EOF packet has already been received, but that function also checks that the read data queue is empty before reporting the EOF. That caused channel_wait_closed to fail with a LIBSSH2_ERROR_INVAL when some data was queued even after a successful call to libssh2_channel_wait_eof. This patch changes libssh2_channel_wait_closed to look directly into channel->remote.eof so that both libssh2_channel_wait_eof and libssh2_channel_wait_closed bahave consistently. - [Salvador Fandino brought this change] channel_wait_eof: fix debug message Daniel Stenberg (25 Oct 2016) - libssh2.h: start working on 1.8.1 Version 1.8.0 (25 Oct 2016) Daniel Stenberg (25 Oct 2016) - RELEASE-NOTES: adjusted for 1.8.0 Kamil Dudka (20 Oct 2016) - Revert "aes: the init function fails when OpenSSL has AES support" This partially reverts commit f4f2298ef3635acd031cc2ee0e71026cdcda5864 because it caused the compatibility code to call initialization routines redundantly, leading to memory leakage with OpenSSL 1.1 and broken curl test-suite in Fedora: 88 bytes in 1 blocks are definitely lost in loss record 5 of 8 at 0x4C2DB8D: malloc (vg_replace_malloc.c:299) by 0x72C607D: CRYPTO_zalloc (mem.c:100) by 0x72A2480: EVP_CIPHER_meth_new (cmeth_lib.c:18) by 0x4E5A550: make_ctr_evp.isra.0 (openssl.c:407) by 0x4E5A8E8: _libssh2_init_aes_ctr (openssl.c:471) by 0x4E5BB5A: libssh2_init (global.c:49) Daniel Stenberg (19 Oct 2016) - [Charles Collicutt brought this change] libssh2_wait_socket: Fix comparison with api_timeout to use milliseconds (#134) Fixes #74 - [Charles Collicutt brought this change] Set err_msg on _libssh2_wait_socket errors (#135) - Revert "travis: Test mbedtls too" This reverts commit 3e6de50a24815e72ec5597947f1831f6083b7da8. Travis doesn't seem to support the mbedtls-dev package - maketgz: support "only" to only update version number locally and fix the date output locale - configure: make the --with-* options override the OpenSSL default ... previously it would default to OpenSSL even with the --with-[crypto] options used unless you specificly disabled OpenSSL. Now, enabling another backend will automatically disable OpenSSL if the other one is found. - [Keno Fischer brought this change] docs: Add documentation on new cmake/configure options - [Keno Fischer brought this change] configure: Add support for building with mbedtls - [wildart brought this change] travis: Test mbedtls too - [wildart brought this change] crypto: add support for the mbedTLS backend Closes #132 - [wildart brought this change] cmake: Add CLEAR_MEMORY option, analogously to that for autoconf - README.md: fix link typo - README: markdown version to look nicer on github Viktor Szakats (5 Sep 2016) - [Taylor Holberton brought this change] openssl: add OpenSSL 1.1.0 compatibility Daniel Stenberg (4 Sep 2016) - [Antenore Gatta brought this change] tests: HAVE_NETINET_IN_H was not defined correctly (#127) Fixes #125 - SECURITY: fix web site typo - SECURITY: security process GitHub (14 Aug 2016) - [Alexander Lamaison brought this change] Basic dockerised test suite. This introduces a test suite for libssh2. It runs OpenSSH in a Docker container because that works well on Windows (via docker-machine) as well as Linux. Presumably it works on Mac too with docker-machine, but I've not tested that. Because the test suite is docker-machine aware, you can also run it against a cloud provider, for more realistic network testing, by setting your cloud provider as your active docker machine. The Appveyor CI setup in this commit does that because Appveyor doesn't support docker locally. Kamil Dudka (3 Aug 2016) - [Viktor Szakats brought this change] misc.c: Delete unused static variables Closes #114 Daniel Stenberg (9 Apr 2016) - [Will Cosgrove brought this change] Merge pull request #103 from willco007/patch-2 Fix for security issue CVE-2016-0787 Alexander Lamaison (2 Apr 2016) - [Zenju brought this change] Fix MSVC 14 compilation errors For _MSC_VER == 1900 these macros are not needed and create problems: 1>C:\Program Files (x86)\Windows Kits\10\Include\10.0.10240.0\ucrt\stdio.h(1925): warning C4005: 'snprintf': macro redefinition (compiling source file libssh2-files\src\mac.c) 1> \win32\libssh2_config.h(27): note: see previous definition of 'snprintf' (compiling source file libssh2-files\src\mac.c) 1>C:\Program Files (x86)\Windows Kits\10\Include\10.0.10240.0\ucrt\stdio.h(1927): fatal error C1189: #error: Macro definition of snprintf conflicts with Standard Library function declaration (compiling source file libssh2-files\src\mac.c) Daniel Stenberg (26 Mar 2016) - [Brad Harder brought this change] _libssh2_channel_open: speeling error fixed in channel error message Alexander Lamaison (15 Mar 2016) - Link with crypt32.lib on Windows. Makes linking with static OpenSSL work again. Although it's not required for dynamic OpenSSL, it does no harm. Fixes #98. - [Craig A. Berry brought this change] Tweak VMS help file building. Primarily this is handling cases where top-level files moved into the docs/ directory. I also corrected a typo and removed the claim that libssh2 is public domain. - [Craig A. Berry brought this change] Build with standard stat structure on VMS. This gets us large file support, is available on any VMS release in the last decade and more, and gives stat other modern features such as 64-bit ino_t. - [Craig A. Berry brought this change] Update vms/libssh2_config.h. VMS does have stdlib.h, gettimeofday(), and OpenSSL. The latter is appropriate to hard-wire in the configuration because it's installed by default as part of the base operating system and there is currently no libgcrypt port. - [Craig A. Berry brought this change] VMS can't use %zd for off_t format. %z is a C99-ism that VMS doesn't currently have; even though the compiler is C99-compliant, the library isn't quite. The off_t used for the st_size element of the stat can be 32-bit or 64-bit, so detect what we've got and pick a format accordingly. - [Craig A. Berry brought this change] Normalize line endings in libssh2_sftp_get_channel.3. Somehow it got Windows-style CRLF endings so convert to just LF, for consistency as well as not to confuse tools that will regard the \r as content (e.g. the OpenVMS help librarian). Dan Fandrich (29 Feb 2016) - libgcrypt: Fixed a NULL pointer dereference on OOM Daniel Stenberg (24 Feb 2016) - [Viktor Szakats brought this change] url updates, HTTP => HTTPS Closes #87 Dan Fandrich (23 Feb 2016) - RELEASE-NOTES: removed some duplicated names Version 1.7.0 (23 Feb 2016) Daniel Stenberg (23 Feb 2016) - web: the site is now HTTPS - RELEASE-NOTES: 1.7.0 release - diffie_hellman_sha256: convert bytes to bits As otherwise we get far too small numbers. Reported-by: Andreas Schneider CVE-2016-0787 Alexander Lamaison (18 Feb 2016) - Allow CI failures with VS 2008 x64. Appveyor doesn't support this combination. Daniel Stenberg (16 Feb 2016) - [Viktor Szakats brought this change] GNUmakefile: list system libs after user libs Otherwise some referenced WinSock functions will fail to resolve when linking against LibreSSL 2.3.x static libraries with mingw. Closes #80 - [Viktor Szakats brought this change] openssl: apply new HAVE_OPAQUE_STRUCTS macro Closes #81 - [Viktor Szakats brought this change] openssl: fix LibreSSL support after OpenSSL 1.1.0-pre1/2 support Alexander Lamaison (14 Feb 2016) - sftp.h: Fix non-C90 type. uint64_t does not exist in C90. Use libssh2_uint64_t instead. - Exclude sshd tests from AppVeyor. They fail complaining that sshd wasn't invoked with an absolute path. - Test on more versions of Visual Studio. - Fix Appveyor builds. Daniel Stenberg (14 Feb 2016) - [Viktor Szakats brought this change] openssl: add OpenSSL 1.1.0-pre3-dev compatibility by using API instead of accessing an internal structure. Closes #83 - RELEASE-NOTES: synced with 996b04ececdf - include/libssh2.h: next version is 1.7.0 - configure: build "silent" if possible - sftp: re-indented some minor stuff - [Jakob Egger brought this change] sftp.c: ensure minimum read packet size For optimum performance we need to ensure we don't request tiny packets. - [Jakob Egger brought this change] sftp.c: Explicit return values & sanity checks - [Jakob Egger brought this change] sftp.c: Check Read Packet File Offset This commit adds a simple check to see if the offset of the read request matches the expected file offset. We could try to recover, from this condition at some point in the future. Right now it is better to return an error instead of corrupted data. - [Jakob Egger brought this change] sftp.c: Don't return EAGAIN if data was written to buffer - [Jakob Egger brought this change] sftp.c: Send at least one read request before reading This commit ensures that we have sent at least one read request before we try to read data in sftp_read(). Otherwise sftp_read() would return 0 bytes (indicating EOF) if the socket is not ready for writing. - [Jakob Egger brought this change] sftp.c: stop reading when buffer is full Since we can only store data from a single chunk in filep, we have to stop receiving data as soon as the buffer is full. This adresses the following bug report: https://github.com/libssh2/libssh2/issues/50 Salvador Fandiño (21 Jan 2016) - agent_disconnect_unix: unset the agent fd after closing it "agent_disconnect_unix", called by "libssh2_agent_disconnect", was leaving the file descriptor in the agent structure unchanged. Later, "libssh2_agent_free" would call again "libssh2_agent_disconnect" under the hood and it would try to close again the same file descriptor. In most cases that resulted in just a harmless error, but it is also possible that the file descriptor had been reused between the two calls resulting in the closing of an unrelated file descriptor. This patch sets agent->fd to LIBSSH2_INVALID_SOCKET avoiding that issue. Signed-off-by: Salvador Fandiño <sfandino@yahoo.com> Daniel Stenberg (18 Jan 2016) - [Patrick Monnerat brought this change] os400qc3: support encrypted private keys PKCS#8 EncryptedPrivateKeyinfo structures are recognized and decoded to get values accepted by the Qc3 crypto library. - [Patrick Monnerat brought this change] os400qc3: New PKCS#5 decoder The Qc3 library is not able to handle PKCS#8 EncryptedPrivateKeyInfo structures by itself. It is only capable of decrypting the (encrypted) PrivateKeyInfo part, providing a key encryption key and an encryption algorithm are given. Since the encryption key and algorithm description part in a PKCS#8 EncryptedPrivateKeyInfo is a PKCS#5 structure, such a decoder is needed to get the derived key method and hash, as well as encryption algorith and initialisation vector. - [Patrick Monnerat brought this change] os400qc3: force continuous update on non-final hash/hmac computation - [Patrick Monnerat brought this change] os400qc3: Be sure hmac keys have a minimum length The Qc3 library requires a minimum key length depending on the target hash algorithm. Append binary zeroes to the given key if not long enough. This matches RFC 2104 specifications. - [Patrick Monnerat brought this change] os400qc3: Slave descriptor for key encryption key The Qc3 library requires the key encryption key to exist as long as the encrypted key is used. Its descriptor token is then kept as an "encrypted key slave" for recursive release. - [Patrick Monnerat brought this change] os400qc3.c: comment PEM/DER decoding - [Patrick Monnerat brought this change] os400qc3.c: improve ASN.1 header byte checks - [Patrick Monnerat brought this change] os400qc3.c: improve OID matching - [Patrick Monnerat brought this change] os400: os400qc3.c: replace malloc by LIBSSH2_ALLOC or alloca where possible - [Patrick Monnerat brought this change] os400: asn1_new_from_bytes(): use data from a single element only - [Patrick Monnerat brought this change] os400: fix an ILE/RPG prototype - [Patrick Monnerat brought this change] os400: implement character encoding conversion support - [Patrick Monnerat brought this change] os400: do not miss some external prototypes Build procedure extproto() did not strip braces from header files, thus possibly prepended them to true prototypes. This prevented the prototype to be recognized as such. The solution implemented here is to map braces to semicolons, effectively considering them as potential prototype delimiters. - [Patrick Monnerat brought this change] os400: Really add specific README - [Patrick Monnerat brought this change] os400: Add specific README and include new files in dist tarball - [Patrick Monnerat brought this change] os400: add compilation scripts - [Patrick Monnerat brought this change] os400: include files for ILE/RPG In addition, file os400/macros.h declares all procedures originally defined as macros. It must not be used for real inclusion and is only intended to be used as a `database' for macro wrapping procedures generation. - [Patrick Monnerat brought this change] os400: add supplementary header files/wrappers. Define configuration. - [Patrick Monnerat brought this change] Protect callback function calls from macro substitution Some structure fields holding callback addresses have the same name as the underlying system function (connect, send, recv). Set parentheses around their reference to suppress a possible macro substitution. Use a macro for connect() on OS/400 to resolve a const/nonconst parameter problem. - [Patrick Monnerat brought this change] Add interface for OS/400 crypto library QC3 - [Patrick Monnerat brought this change] misc: include stdarg.h for debug code - [Patrick Monnerat brought this change] Document crypto library interface - [Patrick Monnerat brought this change] Feature an optional crypto-specific macro to rsa sign a data fragment vector OS/400 crypto library is unable to sign a precomputed SHA1 hash: however it does support a procedure that hashes data fragments and rsa signs. If defined, the new macro _libssh2_rsa_sha1_signv() implements this function and disables use of _libssh2_rsa_sha1_sign(). The function described above requires that the struct iovec unused slacks are cleared: for this reason, macro libssh2_prepare_iovec() has been introduced. It should be defined as empty for crypto backends that are not sensitive to struct iovec unused slack values. - [Patrick Monnerat brought this change] Fold long lines in include files - [Viktor Szakats brought this change] kex.c: fix indentation Closes #71 - [Viktor Szakats brought this change] add OpenSSL-1.1.0-pre2 compatibility Closes #70 - [Viktor Szakats brought this change] add OpenSSL 1.1.0-pre1 compatibility * close https://github.com/libssh2/libssh2/issues/69 * sync a declaration with the rest of similar ones * handle EVP_MD_CTX_new() returning NULL with OpenSSL 1.1.0 * fix potential memory leak with OpenSSL 1.1.0 in _libssh2_*_init() functions, when EVP_MD_CTX_new() succeeds, but EVP_DigestInit() fails. Marc Hoersken (22 Dec 2015) - wincng.c: fixed _libssh2_wincng_hash_final return value _libssh2_wincng_hash_final was returning the internal BCRYPT status code instead of a valid libssh2 return value (0 or -1). This also means that _libssh2_wincng_hash never returned 0. - wincng.c: fixed possible memory leak in _libssh2_wincng_hash If _libssh2_wincng_hash_update failed _libssh2_wincng_hash_final would never have been called before. Reported by Zenju. Kamil Dudka (15 Dec 2015) - [Paul Howarth brought this change] libssh2.pc.in: fix the output of pkg-config --libs ... such that it does not include LDFLAGS used to build libssh2 itself. There was a similar fix in the curl project long time ago: https://github.com/bagder/curl/commit/curl-7_19_7-56-g4c8adc8 Bug: https://bugzilla.redhat.com/1279966 Signed-off-by: Kamil Dudka <kdudka@redhat.com> Marc Hoersken (6 Dec 2015) - hostkey.c: align code path of ssh_rsa_init to ssh_dss_init - hostkey.c: fix invalid memory access if libssh2_dsa_new fails Reported by dimmaq, fixes #66 Daniel Stenberg (3 Nov 2015) - [Will Cosgrove brought this change] gcrypt: define libssh2_sha256_ctx Looks like it didn't make it into the latest commit for whatever reason. Closes #58 - [Salvador Fandino brought this change] libssh2_session_set_last_error: Add function Net::SSH2, the Perl wrapping module for libssh2 implements several features* on top of libssh2 that can fail and so need some mechanism to report the error condition to the user. Until now, besides the error state maintained internally by libssh2, another error state was maintained at the Perl level for every session object and then additional logic was used to merge both error states. That is a maintenance nighmare, and actually there is no way to do it correctly and consistently. In order to allow the high level language to add new features to the library but still rely in its error reporting features the new function libssh2_session_set_last_error (that just exposses _libssh2_error_flags) is introduced. *) For instance, connecting to a remote SSH service giving the hostname and port. Signed-off-by: Salvador Fandino <sfandino@yahoo.com> Signed-off-by: Salvador Fandiño <sfandino@yahoo.com> - [Salvador Fandino brought this change] _libssh2_error: Support allocating the error message Before this patch "_libssh2_error" required the error message to be a static string. This patch adds a new function "_libssh2_error_flags" accepting an additional "flags" argument and specifically the flag "LIBSSH2_ERR_FLAG_DUP" indicating that the passed string must be duplicated into the heap. Then, the method "_libssh2_error" has been rewritten to use that new function under the hood. Signed-off-by: Salvador Fandino <sfandino@yahoo.com> Signed-off-by: Salvador Fandiño <sfandino@yahoo.com> - [Will Cosgrove brought this change] added engine.h include to fix warning - [sune brought this change] kex.c: removed dupe entry from libssh2_kex_methods[] Closes #51 - [Salvador Fandiño brought this change] userauth: Fix off by one error when reading public key file After reading the public key from file the size was incorrectly decremented by one. This was usually a harmless error as the last character on the public key file is an unimportant EOL. But if due to some error the public key file is empty, the public key size becomes (uint)(0 - 1), resulting in an unrecoverable out of memory error later. Signed-off-by: Salvador Fandi??o <sfandino-/E1597aS9LQAvxtiuMwx3w@public.gmane.org> - [Salvador Fandino brought this change] channel: Detect bad usage of libssh2_channel_process_startup A common novice programmer error (at least among those using the wrapping Perl module Net::SSH2), is to try to reuse channels. This patchs detects that incorrect usage and fails with a LIBSSH2_ERROR_BAD_USE error instead of hanging. Signed-off-by: Salvador Fandino <sfandino-/E1597aS9LQAvxtiuMwx3w@public.gmane.org> - [Will Cosgrove brought this change] kex: Added diffie-hellman-group-exchange-sha256 support ... and fixed HMAC_Init depricated usage Closes #48 Alexander Lamaison (21 Sep 2015) - Prefixed new #defines to prevent collisions. Other libraries might have their own USE_WIN32_*FILES. - [keith-daigle brought this change] Update examples/scp.c to fix bug where large files on win32 would cause got to wrap and go negative - [David Byron brought this change] add libssh2_scp_recv2 to support large (> 2GB) files on windows Daniel Stenberg (17 Sep 2015) - [sune brought this change] WinCNG: support for SHA256/512 HMAC Closes #47 - [brian m. carlson brought this change] Add support for HMAC-SHA-256 and HMAC-SHA-512. Implement support for these algorithms and wire them up to the libgcrypt and OpenSSL backends. Increase the maximum MAC buffer size to 64 bytes to prevent buffer overflows. Prefer HMAC-SHA-256 over HMAC-SHA-512, and that over HMAC-SHA-1, as OpenSSH does. Closes #40 - [Zenju brought this change] kex: free server host key before allocating it (again) Fixes a memory leak when Synology server requests key exchange Closes #43 - [Viktor Szakats brought this change] GNUmakefile: up OpenSSL version closes #23 - [Viktor Szakats brought this change] GNUmakefile: add -m64 CFLAGS when targeting mingw64, add -m32/-m64 to LDFLAGS libssh2 equivalent of curl patch https://github.com/bagder/curl/commit/d21b66835f2af781a3c2a685abc92ef9f0cd86be This allows to build for the non-default target when using a multi-target mingw distro. Also bump default OpenSSL dependency path to 1.0.2c. - [Viktor Szakats brought this change] GNUmakefile: add support for LIBSSH2_LDFLAG_EXTRAS It is similar to existing LIBSSH2_CFLAG_EXTRAS, but for extra linker options. Also delete some line/file ending whitespace. closes #27 - [nasacj brought this change] hostkey.c: Fix compiling error when OPENSSL_NO_MD5 is defined Closes #32 - [Mizunashi Mana brought this change] openssl.h: adjust the rsa/dsa includes ... to work when built without DSA support. Closes #36 Alexander Lamaison (26 Jul 2015) - Let CMake build work as a subproject. Patch contributed by JasonHaslam. - Fix builds with Visual Studio 2015. VS2015 moved stdio functions to the header files as inline function. That means check_function_exists can't detect them because it doesn't use header files - just does a link check. Instead we need to use check_symbol_exists with the correct headers. Kamil Dudka (2 Jul 2015) - cmake: include CMake files in the release tarballs Despite we announced the CMake support in libssh2-1.6.0 release notes, the files required by the CMake build system were not included in the release tarballs. Hence, the only way to use CMake for build was the upstream git repository. This commit makes CMake actually supported in the release tarballs. - tests/mansyntax.sh: fix 'make distcheck' with recent autotools Do not create symbolic links off the build directory. Recent autotools verify that out-of-source build works even if the source directory tree is not writable. - openssl: fix memleak in _libssh2_dsa_sha1_verify() Daniel Stenberg (12 Jun 2015) - openssl: make libssh2_sha1 return error code - use the internal prefix _libssh2_ for non-exported functions - removed libssh2_md5() since it wasn't used Reported-by: Kamil Dudka - [LarsNordin-LNdata brought this change] SFTP: Increase speed and datasize in SFTP read The function sftp_read never return more then 2000 bytes (as it should when I asked Daniel). I increased the MAX_SFTP_READ_SIZE to 30000 but didn't get the same speed as a sftp read in SecureSSH. I analyzed the code and found that a return always was dona when a chunk has been read. I changed it to a sliding buffer and worked on all available chunks. I got an increase in speed and non of the test I have done has failed (both local net and over Internet). Please review and test. I think 30000 is still not the optimal MAX_SFTP_READ_SIZE, my next goal is to make an API to enable changing this value (The SecureSSH sftp_read has more complete filled packages when comparing the network traffic) - bump: start working on 1.6.1 Version 1.6.0 (5 Jun 2015) Daniel Stenberg (5 Jun 2015) - RELEASE-NOTES: synced with 858930cae5c6a Marc Hoersken (19 May 2015) - wincng.c: fixed indentation - [sbredahl brought this change] wincng.c: fixed memleak in (block) cipher destructor Alexander Lamaison (6 May 2015) - [Jakob Egger brought this change] libssh2_channel_open: more detailed error message The error message returned by libssh2_channel_open in case of a server side channel open failure is now more detailed and includes the four standard error conditions in RFC 4254. - [Hannes Domani brought this change] kex: fix libgcrypt memory leaks of bignum Fixes #168. Marc Hoersken (3 Apr 2015) - configure.ac: check for SecureZeroMemory for clear memory feature - Revert "wincng.c: fix clear memory feature compilation with mingw" This reverts commit 2d2744efdd0497b72b3e1ff6e732aa4c0037fc43. Autobuilds show that this did not solve the issue. And it seems like RtlFillMemory is defined to memset, which would be optimized out by some compilers. - wincng.c: fix clear memory feature compilation with mingw Alexander Lamaison (1 Apr 2015) - [LarsNordin-LNdata brought this change] Enable use of OpenSSL that doesn't have DSA. Added #if LIBSSH2_DSA for all DSA functions. - [LarsNordin-LNdata brought this change] Use correct no-blowfish #define with OpenSSL. The OpenSSL define is OPENSSL_NO_BF, not OPENSSL_NO_BLOWFISH. Marc Hoersken (25 Mar 2015) - configure: error if explicitly enabled clear-memory is not supported This takes 22bd8d81d8fab956085e2079bf8c29872455ce59 and b8289b625e291bbb785ed4add31f4759241067f3 into account, but still makes it enabled by default if it is supported and error out in case it is unsupported and was requested. Daniel Stenberg (25 Mar 2015) - configure: make clear-memory default but only WARN if backend unsupported ... instead of previous ERROR. Marc Hoersken (24 Mar 2015) - wincng.h: fix warning about computed return value not being used - nonblocking examples: fix warning about unused tvdiff on Mac OS X Daniel Stenberg (24 Mar 2015) - openssl: fix compiler warnings - cofigure: fix --disable-clear-memory check Marc Hoersken (23 Mar 2015) - scp.c: improved command length calculation Reduced number of calls to strlen, because shell_quotearg already returns the length of the resulting string (e.q. quoted path) which we can add to the existing and known cmd_len. Removed obsolete call to memset again, because we can put a final NULL-byte at the end of the string using the calculated length. - scp.c: improved and streamlined formatting - scp.c: fix that scp_recv may transmit not initialised memory - scp.c: fix that scp_send may transmit not initialised memory Fixes ticket 244. Thanks Torsten. - kex: do not ignore failure of libssh2_sha1_init() Based upon 43b730ce56f010e9d33573fcb020df49798c1ed8. Fixes ticket 290. Thanks for the suggestion, mstrsn. - wincng.h: fix return code of libssh2_md5_init() - openssl.c: fix possible segfault in case EVP_DigestInit fails - wincng.c: fix possible use of uninitialized variables - wincng.c: fix unused argument warning if clear memory is not enabled - wincng: Added explicit clear memory feature to WinCNG backend This re-introduces the original feature proposed during the development of the WinCNG crypto backend. It still needs to be added to libssh2 itself and probably other backends. Memory is cleared using the function SecureZeroMemory which is available on Windows systems, just like the WinCNG backend. - wincng.c: fixed mixed line-endings - wincng.c: fixed use of invalid parameter types in a8d14c5dcf - wincng.c: only try to load keys corresponding to the algorithm - wincng.c: moved PEM headers into definitions - wincng.h: fixed invalid parameter name - wincng: fixed mismatch with declarations in crypto.h - userauth.c: fixed warning C6001: using uninitialized sig and sig_len - pem.c: fixed warning C6269: possible incorrect order of operations - wincng: add support for authentication keys to be passed in memory Based upon 18cfec8336e and daa2dfa2db. - pem.c: add _libssh2_pem_parse_memory to parse PEM from memory Requirement to implement 18cfec8336e for Libgcrypt and WinCNG. - pem.c: fix copy and paste mistake from 55d030089b8 - userauth.c: fix another possible dereference of a null pointer - userauth.c: fix possible dereference of a null pointer - pem.c: reduce number of calls to strlen in readline Alexander Lamaison (17 Mar 2015) - [Will Cosgrove brought this change] Initialise HMAC_CTX in more places. Missed a couple more places we init ctx to avoid openssl threading crash. - Build build breakage in WinCNG backend caused when adding libssh2_userauth_publickey_frommemory. The new feature isn't implemented for the WinCNG backend currently, but the WinCNG backend didn't contain any implementation of the required backend functions - even ones that returns an error. That caused link errors. This change fixes the problem by providing an implementation of the backend functions that returns an error. - Fix breakage in WinCNG backend caused by introducing libssh2_hmac_ctx_init. The macro was defined to nothing for the libgcrypt backend, but not for WinCNG. This brings the latter into line with the former. Daniel Stenberg (15 Mar 2015) - userauth_publickey_frommemory.3: add AVAILABILITY ... it will be added in 1.6.0 - libssh2: next version will be called 1.6.0 ... since we just added a new function. - docs: add libssh2_userauth_publickey_frommemory.3 to dist The function and man page were added in commit 18cfec8336e - [Jakob Egger brought this change] direct_tcpip: Fixed channel write There were 3 bugs in this loop: 1) Started from beginning after partial writes 2) Aborted when 0 bytes were sent 3) Ignored LIBSSH2_ERROR_EAGAIN See also: https://trac.libssh2.org/ticket/281 https://trac.libssh2.org/ticket/293 Alexander Lamaison (15 Mar 2015) - [Will Cosgrove brought this change] Must init HMAC_CTX before using it. Must init ctx before using it or openssl will reuse the hmac which is not thread safe and causes a crash. Added libssh2_hmac_ctx_init macro. - Add continuous integration configurations. Linux-based CI is done by Travis CI. Windows-based CI is done by Appveyor. - [David Calavera brought this change] Allow authentication keys to be passed in memory. All credits go to Joe Turpin, I'm just reaplying and cleaning his patch: http://www.libssh2.org/mail/libssh2-devel-archive-2012-01/0015.shtml * Use an unimplemented error for extracting keys from memory with libgcrypt. Daniel Stenberg (14 Mar 2015) - docs: include the renamed INSTALL* files in dist Alexander Lamaison (13 Mar 2015) - Prevent collisions between CMake and Autotools in examples/ and tests/. - Avoid clash between CMake build and Autotools. Autotools expects a configuration template file at src/libssh2_config.h.in, which buildconf generates. But the CMake build system has its CMake-specific version of the file at this path. This means that, if you don't run buildconf, the Autotools build will fail because it configured the wrong header template. See https://github.com/libssh2/libssh2/pull/8. - Merge pull request #8 from alamaison/cmake CMake build system. - CMake build system. Tested: - Windows: - Visual C++ 2005/2008/2010/2012/2013/MinGW-w64 - static/shared - 32/64-bit - OpenSSL/WinCNG - Without zlib - Linux: - GCC 4.6.3/Clang 3.4 - static/shared - 32/64-bit - OpenSSL/Libgcrypt - With/Without zlib - MacOS X - AppleClang 6.0.0 - static - 64-bit - OpenSSL - Without zlib Conflicts: README - Man man syntax tests fail gracefully if man version is not suitable. - Return valid code from test fixture on failure. The sshd test fixture was returning -1 if an error occurred, but negative error codes aren't technically valid (google it). Bash on Windows converted them to 0 which made setup failure look as though all tests were passing. - Let mansyntax.sh work regardless of where it is called from. Daniel Stenberg (12 Mar 2015) - [Viktor Szakáts brought this change] mingw build: allow to pass custom CFLAGS Allow to pass custom `CFLAGS` options via environment variable `LIBSSH2_CFLAG_EXTRAS`. Default and automatically added options of `GNUmakefile` have preference over custom ones. This addition is useful for passing f.e. custom CPU tuning or LTO optimization (`-flto -ffat-lto-objects`) options. The only current way to do this is to edit `GNUmakefile`. This patch makes it unnecessary. This is a mirror of similar libcurl patch: https://github.com/bagder/curl/pull/136 - [Will Cosgrove brought this change] userauth: Fixed prompt text no longer being copied to the prompts struct Regression from 031566f9c - README: update the git repo locations - wait_socket: wrong use of difftime() With reversed arguments it would always return a negative value... Bug: https://github.com/bagder/libssh2/issues/1 - bump: start working toward 1.5.1 now Version 1.5.0 (11 Mar 2015) Daniel Stenberg (11 Mar 2015) - RELEASE-NOTES: 1.5.0 release - [Mariusz Ziulek brought this change] kex: bail out on rubbish in the incoming packet CVE-2015-1782 Bug: http://www.libssh2.org/adv_20150311.html - docs: move INSTALL, AUTHORS, HACKING and TODO to docs/ And with this, cleanup README to be shorter and mention the new source code home. - .gitignore: don't ignore INSTALL Dan Fandrich (4 Mar 2015) - examples/x11.c: include sys/select.h for improved portability Daniel Stenberg (4 Mar 2015) - RELEASE-NOTES: synced with a8473c819bc068 In preparation for the upcoming 1.5.0 release. Guenter Knauf (8 Jan 2015) - NetWare build: added some missing exports. Marc Hoersken (29 Dec 2014) - knownhost.c: fix use of uninitialized argument variable wrote Detected by clang scan in line 1195, column 18. - examples/x11.c: fix result of operation is garbage or undefined Fix use of uninitialized structure w_size_bck. Detected by clang scan in line 386, column 28. - examples/x11.c: remove dead assigments of some return values Detected by clang scan in line 212, column 9. Detected by clang scan in line 222, column 13. Detected by clang scan in line 410, column 13. - examples/x11.c: fix possible memory leak if read fails Detected by clang scan in line 224, column 21. - examples/x11.c: fix invalid removal of first list element Fix use of memory after it was being freed. Detected by clang scan in line 56, column 12. - userauth.c: make sure that sp_len is positive and avoid overflows ... if the pointer subtraction of sp1 - pubkey - 1 resulted in a negative or larger value than pubkey_len, memchr would fail. Reported by Coverity CID 89846. - channel.c: remove logically dead code, host cannot be NULL here ... host cannot be NULL in line 525, because it is always valid (e.g. at least set to "0.0.0.0") after lines 430 and 431. Reported by Coverity CID 89807. - session.c: check return value of session_nonblock during startup Reported by Coverity CID 89803. - session.c: check return value of session_nonblock in debug mode Reported by Coverity CID 89805. - pem.c: fix mixed line-endings introduced with 8670f5da24 - pem.c: make sure there's a trailing zero and b64data is not NULL ... if there is no base64 data between PEM header and footer. Reported by Coverity CID 89823. - kex.c: make sure mlist is not set to NULL ... if the currently unsupported LANG methods are called. Reported by Coverity CID 89834. - packet.c: i < 256 was always true and i would overflow to 0 Visualize that the 0-termination is intentional, because the array is later passed to strlen within _libssh2_packet_askv. - silence multiple data conversion warnings Daniel Stenberg (23 Dec 2014) - agent_connect_unix: make sure there's a trailing zero ... if the path name was too long. Reported by Coverity CID 89801. Marc Hoersken (22 Dec 2014) - examples on Windows: use native SOCKET-type instead of int And check return values accordingly. - userauth.c: improve readability and clarity of for-loops Daniel Stenberg (22 Dec 2014) - calloc: introduce LIBSSH2_CALLOC() A simple function using LIBSSH2_ALLOC + memset, since this pattern was used in multiple places and this simplies code in general. Marc Hoersken (15 Dec 2014) - libssh2_priv.h: Ignore session, context and format parameters - x11 example: check return value of socket function - examples: fixed mixed line-endings introduced with aedfba25b8 - wincng.c: explicitly ignore BCrypt*AlgorithmProvider return codes Fixes VS2012 code analysis warning C6031: return value ignored: <function> could return unexpected value - wincng.c: fix possible invalid memory write access Fixes VS2012 code analysis warning C6386: buffer overrun: accessing 'pbOutput', the writable size is 'cbOutput' bytes, but '3' bytes may be written: libssh2 wincng.c 610 - tests on Windows: check for WSAStartup return code Fixes VS2012 code analysis warning C6031: return value ignored: <function> could return unexpected value - wincng.c: fix possible NULL pointer de-reference of bignum Fixes VS2012 code analysis warning C6011: dereferencing NULL pointer 'bignum'. libssh2 wincng.c 1567 - wincng.c: fix possible use of uninitialized memory Fixes VS2012 code analysis warning C6001: using uninitialized memory 'cbDecoded'. libssh2 wincng.c 553 - packet.c: fix possible NULL pointer de-reference within listen_state Fixes VS2012 code analysis warning C6011: dereferencing NULL pointer 'listen_state->channel'. libssh2 packet.c 221 - kex.c: fix possible NULL pointer de-reference with session->kex Fixes VS2012 code analysis warning C6011: dereferencing NULL pointer 'session->kex'. libssh2 kex.c 1761 - agent.c: check return code of MapViewOfFile Fixes VS2012 code analysis warning C6387: 'p+4' may be '0': this does not adhere to the specification for the function 'memcpy': libssh2 agent.c 330 Fixes VS2012 code analysis warning C6387: 'p' may be '0': this does not adhere to the specification for the function 'UnmapViewOfFile': libssh2 agent.c 333 - examples on Windows: check for socket return code Fixes VS2012 code analysis warning C28193: The variable holds a value that must be examined - examples on Windows: check for WSAStartup return code Fixes VS2012 code analysis warning C6031: return value ignored: <function> could return unexpected value Guenter Knauf (11 Dec 2014) - wincng.c: silent some more gcc compiler warnings. - wincng.c: silent gcc compiler warnings. - Watcom build: added support for WinCNG build. - build: updated dependencies in makefiles. Daniel Stenberg (4 Dec 2014) - configure: change LIBS not LDFLAGS when checking for libs Closes #289 Patch-by: maurerpe Guenter Knauf (3 Dec 2014) - MinGW build: some more GNUMakefile tweaks. test/GNUmakefile: added architecture autodetection; added switches to CFLAGS and RCFLAGS to make sure that the right architecture is used. Added support to build with WinCNG. - sftpdir.c: added authentication method detection. Stuff copied over from ssh2.c to make testing a bit easier. - NMake build: fixed LIBS settings. - NMake build: added support for WinCNG build. - MinGW build: some GNUMakefile tweaks. Added architecture autodetection; added switches to CFLAGS and RCFLAGS to make sure that the right architecture is used. Added support to build with WinCNG. - MinGW build: Fixed redefine warnings. - Updated copyright year. Daniel Stenberg (31 Aug 2014) - COPYING: bump the copyright year Dan Fandrich (28 Jul 2014) - docs: fixed a bunch of typos - docs: added missing libssh2_session_handshake.3 file Marc Hoersken (19 May 2014) - wincng.c: specify the required libraries for dependencies using MSVC Initially reported by Bob Kast as "for MS VS builds, specify the libraries that are required so they don't need to go into all project files that may use this library". Thanks a lot. - [Bob Kast brought this change] windows build: do not export externals from static library If you are building a DLL, then you need to explicitly export each entry point. When building a static library, you should not. libssh2 was exporting the entry points whether it was building a DLL or a static library. To elaborate further, if libssh2 was used as a static library, which was being linked into a DLL, the libssh2 API would be exported from that separate DLL. Daniel Stenberg (19 May 2014) - [Mikhail Gusarov brought this change] Fix typos in manpages Marc Hoersken (18 May 2014) - wincng.c: Fixed memory leak in case of an error during ASN.1 decoding - configure: Display individual crypto backends on separate lines This avoids line-wrapping in between parameters and makes the error message look like the following: configure: error: No crypto library found! Try --with-libssl-prefix=PATH or --with-libgcrypt-prefix=PATH or --with-wincng on Windows - [Bob Kast brought this change] libssh2_priv.h: a 1 bit bit-field should be unsigned some compilers may not like this - knownhost.c: Fixed warning that pointer targets differ in signedness - wincng.c: Fixed warning about pointer targets differing in signedness - tcpip-forward.c: Fixed warning that pointer targets differ in signedness libssh2_channel_forward_listen_ex uses ints instead of unsigned ints. - misc.c: Fixed warning about mixed declarations and code - libgcrypt.h: Fixed warning about pointer targets differing in signedness - wincng.h: Fixed warning about pointer targets differing in signedness - misc.c: Fixed warning about unused parameter abstract - tcpip-forward.c: Removed unused variables shost, sport and sockopt - wincng.h: Added forward declarations for all WinCNG functions Initially reported by Bob Kast as "Wincng - define function prototypes for wincng routines". Thanks a lot. Also replaced structure definitions with type definitions. - [Bob Kast brought this change] libssh2.h: on Windows, a socket is of type SOCKET, not int - win32: Added WinCNG targets to generated Visual Studio project Inspired by Bob Kast's reports, this commit enables the compilation of libssh2 with WinCNG using the generated Visual Studio project files. This commit adds WinCNG support to parts of the existing Win32 build infrastructure, until new build systems, like pre-defined VS project files or CMake files may be added. This commit and b20bfeb3e519119a48509a1099c06d65aa7da1d7 raise one question: How to handle build systems, like VS project files, that need to include all source files regardless of the desired target, including all supported crypto backends? For now the mentioned commit added a check for LIBSSH2_OPENSSL to openssl.c and with this commit the supported crypto backends are hardcoded within Makefile.am. - libssh2_priv msvc: Removed redundant definition of inline keyword Initially reported by Bob Kast as "Remove redundant 'inline' define". Thanks a lot. - wincng: Made data parameter to hash update function constant Initially reported by Bob Kast as "formal parameter must be const since it is used in contexts where the actual parameter may be const". Thanks a lot. - wincng: fix cross-compilation against the w64 mingw-runtime package - openssl: Check for LIBSSH2_OPENSSL in order to compile with openssl - wincng: Fixed use of possible uninitialized variable pPaddingInfo Reported by Bob Kast, thanks a lot. - wincng: Added cast for double to unsigned long conversion - wincng: Cleaned up includes and check NTSTATUS using macro Removed header file combination that is not supported on a real Windows platform and can only be compiled using MinGW. Replaced custom NTSTATUS return code checks with BCRYPT_SUCCESS macro. Daniel Stenberg (16 Mar 2014) - userauth_hostbased_fromfile: zero assign to avoid uninitialized use Detected by clang-analyze - channel_receive_window_adjust: store windows size always Avoid it sometimes returning without storing it, leaving calling functions with unknown content! Detected by clang-analyzer - publickey_packet_receive: avoid junk in returned pointers clang-analyzer found this risk it would return a non-initialized pointer in a success case Peter Stuge (16 Mar 2014) - [Marc Hoersken brought this change] Added Windows Cryptography API: Next Generation based backend - [Marc Hoersken brought this change] knownhost.c: fixed that 'key_type_len' may be used uninitialized ../src/knownhost.c: In function 'libssh2_knownhost_readline': ../src/knownhost.c:651:16: warning: 'key_type_len' may be used uninitialized in this function [-Wmaybe-uninitialized] rc = knownhost_add(hosts, hostbuf, NULL, ^ ../src/knownhost.c:745:12: note: 'key_type_len' was declared here size_t key_type_len; ^ - [Marc Hoersken brought this change] pem.c: always compile pem.c independently of crypto backend - Fix non-autotools builds: Always define the LIBSSH2_OPENSSL CPP macro Commit d512b25f69a1b6778881f6b4b5ff9cfc6023be42 introduced a crypto library abstraction in the autotools build system, to allow us to more easily support new crypto libraries. In that process it was found that all other build system which we support are hard-coded to build with OpenSSL. Commit f5c1a0d98bd51aeb24aca3d49c7c81dcf8bd858d fixes automake introduced into non-autotools build systems but still overlooked the CPP macro saying that we are using OpenSSL. Thanks to Marc Hörsken for identifying this issue and proposing a fix for win32/{GNUmakefile,config.mk}. This commit uses a slightly different approach but the end result is the same. Dan Fandrich (15 Mar 2014) - channel_close: Close the channel even in the case of errors - sftp_close_handle: ensure the handle is always closed Errors are reported on return, but otherwise the close path is completed as much as possible and the handle is freed on exit. Alexander Lamaison (6 Mar 2014) - knownhost: Restore behaviour of `libssh2_knownhost_writeline` with short buffer. Commit 85c6627c changed the behaviour of `libssh2_knownhost_writeline` so that it stopped returning the number of bytes needed when the given buffer was too small. Also, the function changed such that is might write to part of the buffer before realising it is too small. This commit restores the original behaviour, whilst keeping the unknown-key-type functionality that 85c6627c. Instead of writing to the buffer piecemeal, the length of the various parts is calculated up front and the buffer written only if there is enough space. The calculated necessary size is output in `outlen` regardless of whether the buffer was written to. The main use-case for the original behaviour that this commit restores is to allow passing in a NULL buffer to get the actual buffer size needed, before calling the function again with the buffer allocated to the exact size required. - knownhost: Fix DSS keys being detected as unknown. I missing `else` meant ssh-dss format keys were being re-detected as unknown format. Dan Fandrich (6 Mar 2014) - knownhosts: Abort if the hosts buffer is too small This could otherwise cause a match on the wrong host - agent_list_identities: Fixed memory leak on OOM - Fixed a few typos - userauth: Fixed an attempt to free from stack on error - Fixed a few memory leaks in error paths - Fixed two potential use-after-frees of the payload buffer The first might occur if _libssh2_packet_add returns an error, as fullpacket_state wasn't reset to idle so if it were possible for fullpacket to be called again, it would return to the same state handler and re-use the freed p->packet buffer. The second could occur if decrypt returned an error, as it freed the packet buffer but did not clear total_num, meaning that freed buffer could be written into again later. Alexander Lamaison (28 Nov 2013) - Fix missing `_libssh2_error` in `_libssh2_channel_write`. In one case, the error code from `_libssh2_transport_read` was being returned from `_libssh2_channel_write` without setting it as the last error by calling `_libssh2_error`. This commit fixes that. Found when using a session whose socket had been inadvertently destroyed. The calling code got confused because via `libssh2_session_last_error` it appeared no error had occurred, despite one being returned from the previous function. Kamil Dudka (21 Nov 2013) - [Mark McPherson brought this change] openssl: initialise the digest context before calling EVP_DigestInit() When using the OpenSSL libraries in FIPS mode, the function call EVP_DigestInit() is actually #defined to FIPS_digestinit(). Unfortunately wheres EVP_DigestInit() initialises the context and then calls EVP_DigestInit_ex(), this function assumes that the context has been pre-initialised and crashes when it isn't. Bug: https://trac.libssh2.org/ticket/279 Fixes #279 - [Marc Hörsken brought this change] .gitignore: Ignore files like src/libssh2_config.h.in~ Peter Stuge (13 Nov 2013) - Move automake conditionals added by commit d512b25f out of Makefile.inc Commit d512b25f69a1b6778881f6b4b5ff9cfc6023be42 added automake conditionals to Makefile.inc but since Makefile.inc is included from Makefile for all other build systems that does not work. This commit instead adds Makefile.OpenSSL.inc and Makefile.libgcrypt.inc and moves the automake conditional to its proper place, src/Makefile.am. The automake conditional includes the correct Makefile.$name.inc per the crypto library selection/detection done by configure. All non-autotools build system files in libssh2 are hardcoded to use OpenSSL and do not get a conditional but at least there is some reuse because they can all include the new Makefile.OpenSSL.inc. Daniel Stenberg (27 Oct 2013) - [Salvador Fandino brought this change] Set default window size to 2MB The default channel window size used until now was 256KB. This value is too small and results on a bottleneck on real-life networks where round-trip delays can easily reach 300ms. The issue was not visible because the configured channel window size was being ignored and a hard-coded value of ~22MB being used instead, but that was fixed on a previous commit. This patch just changes the default window size (LIBSSH2_CHANNEL_WINDOW_DEFAULT) to 2MB. It is the same value used by OpenSSH and in our opinion represents a good compromise between memory used and transfer speed. Performance tests were run to determine the optimum value. The details and related discussion are available from the following thread on the libssh2 mailing-list: http://www.libssh2.org/mail/libssh2-devel-archive-2013-10/0018.shtml http://article.gmane.org/gmane.network.ssh.libssh2.devel/6543 An excerpt follows: "I have been running some transfer test and measuring their speed. My setup was composed of a quad-core Linux machine running Ubuntu 13.10 x86_64 with a LXC container inside. The data transfers were performed from the container to the host (never crossing through a physical network device). Network delays were simulated using the tc tool. And ping was used to verify that they worked as intended during the tests. The operation performed was the equivalent to the following ssh command: $ ssh container "dd bs=16K count=8K if=/dev/zero" >/dev/null Though, establishment and closing of the SSH connection was excluded from the timings. I run the tests several times transferring files of sizes up to 128MB and the results were consistent between runs. The results corresponding to the 128MB transfer are available here: https://docs.google.com/spreadsheet/ccc?key=0Ao1yRmX6PQQzdG5wSFlrZl9HRWNET3ZyN0hnaGo5ZFE&usp=sharing It clearly shows that 256KB is too small as the default window size. Moving to a 512MB generates a great improvement and after the 1MB mark the returns rapidly diminish. Other factors (TCP window size, probably) become more limiting than the channel window size For comparison I also performed the same transfers using OpenSSH. Its speed is usually on par with that of libssh2 using a window size of 1MB (even if it uses a 2MB window, maybe it is less aggressive sending the window adjust msgs)." Signed-off-by: Salvador Fandino <sfandino@yahoo.com> - [Salvador brought this change] _libssh2_channel_read: Honour window_size_initial _libssh2_channel_read was using an arbitrary hard-coded limit to trigger the window adjusting code. The adjustment used was also hard-coded and arbitrary, 15MB actually, which would limit the usability of libssh2 on systems with little RAM. This patch, uses the window_size parameter passed to libssh2_channel_open_ex (stored as remote.window_size_initial) plus the buflen as the base for the trigger and the adjustment calculation. The memory usage when using the default window size is reduced from 22MB to 256KB per channel (actually, if compression is used, these numbers should be incremented by ~50% to account for the errors between the decompressed packet sizes and the predicted sizes). My tests indicate that this change does not impact the performance of transfers across localhost or a LAN, being it on par with that of OpenSSH. On the other hand, it will probably slow down transfers on networks with high bandwidth*delay when the default window size (LIBSSH2_CHANNEL_WINDOW_DEFAULT=256KB) is used. Signed-off-by: Salvador Fandino <sfandino@yahoo.com> - [Salvador Fandino brought this change] knownhosts: handle unknown key types Store but don't use keys of unsupported types on the known_hosts file. Currently, when libssh2 parses a known_host file containing keys of some type it doesn't natively support, it stops reading the file and returns an error. That means, that the known_host file can not be safely shared with other software supporting other key types (i.e. OpenSSH). This patch adds support for handling keys of unknown type. It can read and write them, even if they are never going to be matched. At the source level the patch does the following things: - add a new unknown key type LIBSSH2_KNOWNHOST_KEY_UNKNOWN - add a new slot (key_type_name) on the known_host struct that is used to store the key type in ascii form when it is not supported - parse correctly known_hosts entries with unknown key types and populate the key_type_name slot - print correctly known_hosts entries of unknown type - when checking a host key ignore keys that do not match the key Fixes #276 - windows build: fix build errors Fixes various link errors with VS2010 Reported-by: "kdekker" Fixes #272 - man page: add missing function argument for libssh2_userauth_publickey_fromfile_ex() Reported-by: "pastey" Fixes #262 - [Salvador brought this change] Fix zlib deflate usage Deflate may return Z_OK even when not all data has been compressed if the output buffer becomes full. In practice this is very unlikely to happen because the output buffer size is always some KBs larger than the size of the data passed for compression from the upper layers and I think that zlib never expands the data so much, even on the worst cases. Anyway, this patch plays on the safe side checking that the output buffer is not exhausted. Signed-off-by: Salvador <sfandino@yahoo.com> - [Salvador brought this change] comp_method_zlib_decomp: Improve buffer growing algorithm The old algorithm was O(N^2), causing lots and lots of reallocations when highly compressed data was transferred. This patch implements a simpler one that just doubles the buffer size everytime it is exhausted. It results in O(N) complexity. Also a smaller inflate ratio is used to calculate the initial size (x4). Signed-off-by: Salvador <sfandino@yahoo.com> - [Salvador brought this change] Fix zlib usage Data may remain in zlib internal buffers when inflate() returns Z_OK and avail_out == 0. In that case, inflate has to be called again. Also, once all the data has been inflated, it returns Z_BUF_ERROR to signal that the input buffer has been exhausted. Until now, the way to detect that a packet payload had been completely decompressed was to check that no data remained on the input buffer but that didn't account for the case where data remained on the internal zlib buffers. That resulted in packets not being completely decompressed and the missing data reappearing on the next packet, though the bug was masked by the buffer allocation algorithm most of the time and only manifested when transferring highly compressible data. This patch fixes the zlib usage. Signed-off-by: Salvador <sfandino@yahoo.com> - [Salvador brought this change] _libssh2_channel_read: fix data drop when out of window After filling the read buffer with data from the read queue, when the window size was too small, "libssh2_channel_receive_window_adjust" was called to increase it. In non-blocking mode that function could return EAGAIN and, in that case, the EAGAIN was propagated upwards and the data already read on the buffer lost. The function was also moving between the two read states "libssh2_NB_state_idle" and "libssh2_NB_state_created" both of which behave in the same way (excepting a debug statment). This commit modifies "_libssh2_channel_read" so that the "libssh2_channel_receive_window_adjust" call is performed first (when required) and if everything goes well, then it reads the data from the queued packets into the read buffer. It also removes the useless "libssh2_NB_state_created" read state. Some rotted comments have also been updated. Signed-off-by: Salvador <sfandino@yahoo.com> - [Salvador Fandino brought this change] window_size: redid window handling for flow control reasons Until now, the window size (channel->remote.window_size) was being updated just after receiving the packet from the transport layer. That behaviour is wrong because the channel queue may grow uncontrolled when data arrives from the network faster that the upper layer consumes it. This patch adds a new counter, read_avail, which keeps a count of the bytes available from the packet queue for reading. Also, now the window size is adjusted when the data is actually read by an upper layer. That way, if the upper layer stops reading data, the window will eventually fill and the remote host will stop sending data. When the upper layers reads enough data, a window adjust packet is delivered and the transfer resumes. The read_avail counter is used to detect the situation when the remote server tries to send data surpassing the window size. In that case, the extra data is discarded. Signed-off-by: Salvador <sfandino@yahoo.com> Peter Stuge (15 Sep 2013) - configure.ac: Call zlib zlib and not libz in text but keep option names - configure.ac: Reorder --with-* options in --help output - configure.ac: Rework crypto library detection This further simplifies adding new crypto libraries. - Clean up crypto library abstraction in build system and source code libssh2 used to explicitly check for libgcrypt and default to OpenSSL. Now all possible crypto libraries are checked for explicitly, making the addition of further crypto libraries both simpler and cleaner. - configure.ac: Add zlib to Requires.private in libssh2.pc if using zlib - Revert "Added Windows Cryptography API: Next Generation based backend" This reverts commit d385230e15715e67796f16f3e65fd899f21a638b. Daniel Stenberg (7 Sep 2013) - [Leif Salomonsson brought this change] sftp_statvfs: fix for servers not supporting statfvs extension Fixes issue arising when server does not support statfvs and or fstatvfs extensions. sftp_statvfs() and sftp_fstatvfs() after this patch will handle the case when SSH_FXP_STATUS is returned from server. - [Marc Hoersken brought this change] Added Windows Cryptography API: Next Generation based backend - [Kamil Dudka brought this change] partially revert "window_size: explicit adjustments only" This partially reverts commit 03ca9020756a4e16f0294e5b35e9826ee6af2364 in order to fix extreme slowdown when uploading to localhost via SFTP. I was able to repeat the issue on RHEL-7 on localhost only. It did not occur when uploading via network and it did not occur on a RHEL-6 box with the same version of libssh2. The problem was that sftp_read() used a read-ahead logic to figure out the window_size, but sftp_packet_read() called indirectly from sftp_write() did not use any read-ahead logic. - _libssh2_channel_write: client spins on write when window full When there's no window to "write to", there's no point in waiting for the socket to become writable since it most likely just will continue to be. Patch-by: ncm Fixes #258 - _libssh2_channel_forward_cancel: avoid memory leaks on error Fixes #257 - _libssh2_packet_add: avoid using uninitialized memory In _libssh2_packet_add, called by _libssh2_packet_read, a call to _libssh2_packet_send that is supposed to send a one-byte message SSH_MSG_REQUEST_FAILURE would send an uninitialized byte upon re-entry if its call to _send returns _EAGAIN. Fixes #259 - _libssh2_channel_forward_cancel: accessed struct after free ... and the assignment was pointless anyway since the struct was about to be freed. Bug introduced in dde2b094. Fixes #268 Peter Stuge (2 Jun 2013) - [Marc Hoersken brought this change] Fixed compilation using mingw-w64 - [Marc Hoersken brought this change] knownhost.c: use LIBSSH2_FREE macro instead of free Use LIBSSH2_FREE instead of free since _libssh2_base64_encode uses LIBSSH2_ALLOC Daniel Stenberg (18 May 2013) - [Matthias Kerestesch brought this change] libssh2_agent_init: init ->fd to LIBSSH2_INVALID_SOCKET ... previously it was left at 0 which is a valid file descriptor! Bug: https://trac.libssh2.org/ticket/265 Fixes #265 - userauth_password: pass on the underlying error code _libssh2_packet_requirev() may return different errors and we pass that to the parent instead of rewriting it. Bug: http://libssh2.org/mail/libssh2-devel-archive-2013-04/0029.shtml Reported by: Cosmin Peter Stuge (9 May 2013) - [Marc Hoersken brought this change] libcrypt.c: Fix typo in _libssh2_rsa_sha1_sign() parameter type Kamil Dudka (4 May 2013) - configure.ac: replace AM_CONFIG_HEADER with AC_CONFIG_HEADERS Reported by: Quintus Bug: https://trac.libssh2.org/ticket/261 Guenter Knauf (12 Apr 2013) - Fixed copyright string for NetWare build. Daniel Stenberg (9 Apr 2013) - [Richard W.M. Jones brought this change] sftp: Add support for fsync (OpenSSH extension). The new libssh2_sftp_fsync API causes data and metadata in the currently open file to be committed to disk at the server. This is an OpenSSH extension to the SFTP protocol. See: https://bugzilla.mindrot.org/show_bug.cgi?id=1798 - [Richard W.M. Jones brought this change] sftp: statvfs: Along error path, reset the correct 'state' variable. - [Richard W.M. Jones brought this change] sftp: seek: Don't flush buffers on same offset Signed-off-by: Richard W.M. Jones <rjones@redhat.com> Guenter Knauf (9 Feb 2013) - Updated dependency libs. - Fixed tool macro names. Daniel Stenberg (29 Nov 2012) - [Seth Willits brought this change] compiler warnings: typecast strlen in macros ... in macro parameters to avoid compiler warnings about lost precision. Several macros in libssh2.h call strlen and pass the result directly to unsigned int parameters of other functions, which warns about precision loss because strlen returns size_t which is unsigned long on at least some platforms (such as OS X). The fix is to simply typecast the strlen() result to unsigned int. - libssh2.h: bump version to 1.4.4-DEV Version 1.4.3 (27 Nov 2012) Daniel Stenberg (27 Nov 2012) - RELEASE-NOTES: fixed for 1.4.3 - sftp_read: return error if a too large package arrives Peter Stuge (13 Nov 2012) - Only define _libssh2_dsa_*() functions when building with DSA support Guenter Knauf (8 Nov 2012) - Added .def file to output. Kamil Dudka (1 Nov 2012) - libssh2_hostkey_hash.3: update the description of return value The function returns NULL also if the hash algorithm is not available. Guenter Knauf (24 Oct 2012) - Fixed mode acciedently committed. - Ignore generated file. - Added hack to make use of Makefile.inc. This should avoid further maintainance of the objects list. - Fixed MSVC NMakefile. Added missing source files; added resource for DLL. Kamil Dudka (22 Oct 2012) - examples: use stderr for messages, stdout for data Reported by: Karel Srot Bug: https://bugzilla.redhat.com/867462 - openssl: do not leak memory when handling errors ,.. in aes_ctr_init(). Detected by Coverity. - channel: fix possible NULL dereference ... in libssh2_channel_get_exit_signal(). Detected by Coverity. - Revert "aes: the init function fails when OpenSSL has AES support" This partially reverts commit f4f2298ef3635acd031cc2ee0e71026cdcda5864. We need to use the EVP_aes_???_ctr() functions in FIPS mode. - crypt: use hard-wired cipher block sizes consistently - openssl: do not ignore failure of EVP_CipherInit() - kex: do not ignore failure of libssh2_md5_init() The MD5 algorithm is disabled when running in FIPS mode. Daniel Stenberg (21 Aug 2012) - [Peter Krempa brought this change] known_hosts: Fail when parsing unknown keys in known_hosts file. libssh2_knownhost_readfile() silently ignored problems when reading keys in unsupported formats from the known hosts file. When the file is written again from the internal structures of libssh2 it gets truntcated to the point where the first unknown key was located. * src/knownhost.c:libssh2_knownhost_readfile() - return error if key parsing fails - AUTHORS: synced with 42fec44c8a4 31 recent authors added - [Dave Hayden brought this change] compression: add support for zlib@openssh.com Add a "use_in_auth" flag to the LIBSSH2_COMP_METHOD struct and a separate "zlib@openssh.com" method, along with checking session->state for LIBSSH2_STATE_AUTHENTICATED. Appears to work on the OpenSSH servers I've tried against, and it should work as before with normal zlib compression. - [Dmitry Smirnov brought this change] configure: gcrypt doesn't come with pkg-config support ... so use plain old -lgcrypt to the linker to link with it. Fixes #225 - sftp_read: Value stored to 'next' is never read Detected by clang-analyzer - publickey_init: errors are negative, fix check Detected by clang-analyzer. - [Maxime Larocque brought this change] session_free: wrong variable used for keeping state If libssh2_session_free is called without the channel being freed previously by libssh2_channel_free a memory leak could occur. A mismatch of states variables in session_free() prevent the call to libssh2_channel_free function. session->state member is used instead of session->free_state. It causes a leak of around 600 bytes on every connection on my systems (Linux, x64 and PPC). (Debugging done under contract for Accedian Networks) Fixes #246 Guenter Knauf (29 Jun 2012) - Small NetWare makefile tweak. - Some small Win32 makefile fixes. Daniel Stenberg (19 Jun 2012) - libssh2_userauth_publickey_fromfile_ex.3: mention publickey == NULL - comp_method_zlib_decomp: handle Z_BUF_ERROR when inflating When using libssh2 to perform an SFTP file transfer from the "JSCAPE MFT Server" (http://www.jscape.com) the transfer failed. The default JSCAPE configuration is to enforce zlib compression on SSH2 sessions so the session was compressed. The relevant part of the debug trace contained: [libssh2] 1.052750 Transport: unhandled zlib error -5 [libssh2] 1.052750 Failure Event: -29 - decompression failure The trace comes from comp_method_zlib_decomp() in comp.c. The "unhandled zlib error -5" is the status returned from the zlib function inflate(). The -5 status corresponds to "Z_BUF_ERROR". The inflate() function takes a pointer to a z_stream structure and "inflates" (decompresses) as much as it can. The relevant fields of the z_stream structure are: next_in - pointer to the input buffer containing compressed data avail_in - the number of bytes available at next_in next_out - pointer to the output buffer to be filled with uncompressed data avail_out - how much space available at next_out To decompress data you set up a z_stream struct with the relevant fields filled in and pass it to inflate(). On return the fields will have been updated so next_in and avail_in show how much compressed data is yet to be processed and next_out and avail_out show how much space is left in the output buffer. If the supplied output buffer is too small then on return there will be compressed data yet to be processed (avail_in != 0) and inflate() will return Z_OK. In this case the output buffer must be grown, avail_out updated and inflate() called again. If the supplied output buffer was big enough then on return the compressed data will have been exhausted (avail_in == 0) and inflate() will return Z_OK, so the data has all been uncompressed. There is a corner case where inflate() makes no progress. That is, there may be unprocessed compressed data and space available in the output buffer and yet the function does nothing. In this case inflate() will return Z_BUF_ERROR. From the zlib documentation and the source code it is not clear under what circumstances this happens. It could be that it needs to write multiple bytes (all in one go) from its internal state to the output buffer before processing the next chunk of input but but can't because there is not enough space (though my guesses as to the cause are not really relevant). Recovery from Z_BUF_ERROR is pretty simple - just grow the output buffer, update avail_out and call inflate() again. The comp_method_zlib_decomp() function does not handle the case when inflate() returns Z_BUF_ERROR. It treats it as a non-recoverable error and basically aborts the session. Fixes #240 Guenter Knauf (12 Jun 2012) - MinGW makefile tweaks. Use GNU tools when compiling on Linux. Fixed dist and dev targets. - NetWare makefile tweaks. Changed to use Windows commandline tools instead of GNU tools when compiling on Windows. Fixed dist and dev targets. Enabled nlmconv error for unresolved symbols. Daniel Stenberg (11 Jun 2012) - Revert "config.rpath: generated file, no need to keep in git" This reverts commit 1ac7bd09cc685755577fb2c8829adcd081e7ab3c. This file still used by lib/*m4 functions so we need to keep the file around. - BINDINGS: added PySsh2, a Python-ctypes binding Guenter Knauf (8 Jun 2012) - Fixed MinGW debug build. Daniel Stenberg (5 Jun 2012) - BINDINGS: Added the Cocoa/Objective-C one ... and sorted the bindings after the languages, alphabetically Reported by: Mike Abdullah - BINDINGS: document the bindings we know of Guenter Knauf (4 Jun 2012) - Fixed LIBSSH2_INT64_T_FORMAT macro. Usually a format macro should hold the whole format, otherwise it should be named a prefix. Also fixed usage of this macro in scp.c for a signed var where it was used as prefix for unsigned. - Removed obsolete define from makefiles. - Renamed NetWare makefiles. - Renamed NetWare makefiles. - Synced MinGW makefiles with 56c64a6..39e438f. Also synced MinGW test makefile with b092696..f8cb874. Peter Stuge (30 May 2012) - Revert "sftp: Don't send attrs.permissions on read-only SSH_FXP_OPEN" This reverts commit 04e79e0c798674a0796be8a55f63dd92e6877790. - sftp: Don't send attrs.permissions on read-only SSH_FXP_OPEN This works around a protocol violation in the ProFTPD 1.3.4 mod_sftp server, as reported by Will Cosgrove in: http://libssh2.org/mail/libssh2-devel-archive-2012-05/0079.shtml Based on a suggested fix by TJ Saunders in: http://libssh2.org/mail/libssh2-devel-archive-2012-05/0104.shtml Guenter Knauf (28 May 2012) - Try to detect OpenSSL build type automatically. Also fixed recently added libgdi32 linkage which is only required when OpenSSL libs are linked statically. Daniel Stenberg (25 May 2012) - config.rpath: generated file, no need to keep in git Guenter Knauf (22 May 2012) - Updated dependency libary versions. Daniel Stenberg (18 May 2012) - 1.4.3: towards the future Version 1.4.2 (18 May 2012) Daniel Stenberg (18 May 2012) - RELEASE-NOTES: synced with 92a9f952794 Alexander Lamaison (15 May 2012) - win32/libssh2_config.h: Remove hardcoded #define LIBSSH2_HAVE_ZLIB. Rationale: Everything else in this file states a fact about the win32 platform that is unconditional for that platform. There is nothing unconditional about the presence of zlib. It is neither included with Windows nor with the platform SDK. Therefore, this is not an appropriate place to assert its presence. Especially as, once asserted, it cannot be overridden using a compiler flag. In contrast, if it is omitted, then it can easily be reasserted by adding a compiler flag defining LIBSSH2_HAVE_ZLIB. Daniel Stenberg (14 May 2012) - RELEASE-NOTES: synced with 69a3354467c - _libssh2_packet_add: SSH_MSG_CHANNEL_REQUEST default to want_reply RFC4254 says the default 'want_reply' is TRUE but the code defaulted to FALSE. Now changed. Fixes #233 - gettimeofday: no need for a replacement under cygwin Fixes #224 Alexander Lamaison (13 May 2012) - Prevent sftp_packet_read accessing freed memory. sftp_packet_add takes ownership of the packet passed to it and (now that we handle zombies) might free the packet. sftp_packet_read uses the packet type byte as its return code but by this point sftp_packet_add might have freed it. This change fixes the problem by caching the packet type before calling sftp_packet_add. I don't understand why sftp_packet_read uses the packet type as its return code. A future change might get rid of this entirely. Daniel Stenberg (12 May 2012) - sftp_packet_flush: flush zombies too As this function is called when the SFTP session is closed, it needs to also kill all zombies left in the SFTP session to avoid leaking memory just in case some zombie would still be in there. - sftp_packetlist_flush: zombies must not have responses already When flushing the packetlist, we must only add the request as a zombie if no response has already been received. Otherwise we could wrongly make it a zombie even though the response was already received and then we'd get a zombie stuck there "forever"... - sftp_read: on EOF remove packet before flush Since the sftp_packetlist_flush() function will move all the existing FXP_READ requests in this handle to the zombie list we must first remove this just received packet as it is clearly not a zombie. - sftp_packet_require: sftp_packet_read() returning 0 is not an error Exactly as the comment in the code said, checking the return code from sftp_packet_read() with <= was wrong and it should be < 0. With the new filtering on incoming packets that are "zombies" we can now see this getting zero returned. - sftp_packetlist_flush: only make it zombie if it was sent The list of outgoing packets may also contain packets that never were sent off and we better not make them zombies too. - [Alexander Lamaison brought this change] Mark outstanding read requests after EOF as zombies. In order to be fast, sftp_read sends many read requests at once. With a small file, this can mean that when EOF is received back, many of these requests are still outstanding. Responses arriving after we close the file and abandon the file handle are queued in the SFTP packet queue and never collected. This causes transfer speed to drop as a progressively longer queue must be searched for every packet. This change introduces a zombie request-ID list in the SFTP session that is used to recognise these outstanding requests and prevent them being added to the queue. Peter Stuge (23 Apr 2012) - [Rafael Kitover brought this change] Update win32/GNUmakefile to use OpenSSL 1.0.1a libcrypto on win32 now depends on gdi32.dll, so move the OpenSSL LDLIBS block to before the compiler definitions, so that libcrypto gets added first, and then add -lgdi32 into the following common LDLIBS for gcc. Guenter Knauf (23 Apr 2012) - Changed 'Requires' to 'Requires.private'. Only static builds need to link against the crypto libs. - Fixed 'Requires:' names. The 'Requires:' line lists the names of the .pc files. - Added 'Requires:' line to libssh2.pc. This is necessary so that other libs which lookup libssh2 info via pkg-config can add the right crypto lib dependencies. - Updated dependency lib versions. Peter Stuge (18 Apr 2012) - configure.ac: Add option to disable build of the example applications Examples are built by default. Any of the following options on the configure command line will skip building them: --disable-examples-build --enable-examples-build=no --enable-examples-build=false - userauth.c: fread() from public key file to correctly detect any errors If the filename parameter for file_read_publickey() was the name of a directory instead of a file then libssh2 would spin trying to fgetc() from the FILE * for the opened directory when trying to determine the length of the encoded public key, since fgetc() can't report errors. Use fread() instead to correctly detect this error condition along with many others. This fixes the problem reported in http://www.libssh2.org/mail/libssh2-devel-archive-2012-04/0021.shtml Reported-by: Oleksiy Zagorskyi <zalex_ua@i.ua> - Return LIBSSH2_ERROR_SOCKET_DISCONNECT on EOF when reading banner Guenter Knauf (17 Apr 2012) - Fixed copyright year. - Updated dependency lib versions in static makefiles. Daniel Stenberg (6 Apr 2012) - version: bump to 1.4.2 We're on the 1.4.2 track now (at least) Version 1.4.1 (4 Apr 2012) Daniel Stenberg (4 Apr 2012) - RELEASE-NOTES: updated for 1.4.1 release - always do "forced" window updates When calling _libssh2_channel_receive_window_adjust() internally, we now always use the 'force' option to prevent libssh2 to avoid sending the update if the update isn't big enough. It isn't fully analyzed but we have seen corner cases which made a necessary window update not get send due to this and then the other side doesn't send data our side then sits waiting for forever. - channel_read: force window adjusts! if there's not enough room to receive the data that's being requested, the window adjustment needs to be sent to the remote and thus the force option has to be used. _libssh2_channel_receive_window_adjust() would otherwise "queue" small window adjustments for a later packet but that is really terribly for the small buffer read that for example is the final little piece of a very large file as then there is no logical next packet! Reported by: Armen Babakhanian Bug: http://www.libssh2.org/mail/libssh2-devel-archive-2012-03/0130.shtml - [Paul Howarth brought this change] aes: the init function fails when OpenSSL has AES support The internal init function only worked fine when the configure script didn't detect the OpenSSL AES_CTR function! Bug: http://www.libssh2.org/mail/libssh2-devel-archive-2012-03/0111.shtml Reported by: Paul Howarth - [Matthew Booth brought this change] transport_send: Finish in-progress key exchange before sending data _libssh2_channel_write() first reads outstanding packets before writing new data. If it reads a key exchange request, it will immediately start key re-exchange, which will require sending a response. If the output socket is full, this will result in a return from _libssh2_transport_read() of LIBSSH2_ERROR_EAGAIN. In order not to block a write because there is no data to read, this error is explicitly ignored and the code continues marshalling a packet for sending. When it is sent, the remote end immediately drops the connection because it was expecting a continuation of the key exchange, but got a data packet. This change adds the same check for key exchange to _libssh2_transport_send() that is in _libssh2_transport_read(). This ensures that key exchange is completed before any data packet is sent. - channel_write: acknowledge transport errors When draining data off the socket with _libssh2_transport_read() (which in turn has to be done so that we can be sure to have read any possible window-increasing packets), this code previously ignored errors which could lead to nasty loops. Now all error codes except EAGAIN will cause the error to be returned at once. Bug: http://www.libssh2.org/mail/libssh2-devel-archive-2012-03/0068.shtml Reported by: Matthew Booth - [Steven Dake brought this change] In examples/x11.c, Make sure sizeof passed to read operation is correct sizeof(buf) expands to 8 or 4 (since its a pointer). This variable may have been static in the past, leading to this error. Signed-off-by: Steven Dake <sdake@redhat.com> - [Steven Dake brought this change] Fix suspicious sizeof usage in examples/x11.c In the x11 example, sizeof(buf) = 8UL (on x86_64), when this should probably represent the buffer size available. I am not sure how to test that this change is actually correct, however. Signed-off-by: Steven Dake <sdake@redhat.com> - sftp_packet_read: follow-up fix for EAGAIN/window adjust The commit in 7194a9bd7ba45 wasn't complete. This change makes sure variables are initialized properly before used in the EAGAIN and window adjust cases. - sftp_packet_add: use named error code instead of number - sftp_packet_add: verify the packet before accepting it In order to bail out as quickly as possible when things are wrong and out of sync, make sure the SFTP message is one we understand. - SFTP: preserve the original error code more Lots of places in the code translated the original error into the more generic LIBSSH2_ERROR_SOCKET_TIMEOUT but this turns out to distort the original error reason a lot and makes tracking down the real origin of a problem really hard. This change makes the original error code be preserved to a larger extent when return up to the parent function. - sftp_packet_read: adjust window size as necessary Commit 03ca9020756 tried to simplify the window sizing logic but broke SFTP readdir as there was no window sizing code left there so large directory listings no longer worked. This change introduces window sizing logic to the sftp_packet_read() function so that it now tells the remote about the local size having a window size that suffice when it is about to ask for directory data. Bug: http://www.libssh2.org/mail/libssh2-devel-archive-2012-03/0069.shtml Reported by: Eric - [Steven Dake brought this change] Tell C compiler we don't care about return code of libssh2_init The call of libssh2_init returns a return code, but nothing could be done within the _libssh2_init_if_needed execution path. Signed-off-by: Steven Dake <sdake@redhat.com> - [Steven Dake brought this change] Add comment indicating a resource leak is not really a resource leak While possibly obvious to those investigating the code, coverity complains about this out of scope leak. Signed-off-by: Steven Dake <sdake@redhat.com> - [Steven Dake brought this change] Use safer snprintf rather then sprintf in scp_send() Signed-off-by: Steven Dake <sdake@redhat.com> - [Steven Dake brought this change] Use safer snprintf rather then sprintf in scp_recv() While the buffer is indeed allocated to a safe length, better safe then sorry. Signed-off-by: Steven Dake <sdake@redhat.com> - [Steven Dake brought this change] use snprintf in knownhost_writeline() rather then sprintf Although the function checks the length, if the code was in error, there could potentially be a buffer overrun with the use of sprintf. Instead replace with snprintf. Signed-off-by: Steven Dake <sdake@redhat.com> - [Steven Dake brought this change] Add tracing to print packets left on session at libssh2_session_free Signed-off-by: Steven Dake <sdake@redhat.com> Peter Stuge (2 Mar 2012) - Define and use LIBSSH2_INVALID_SOCKET instead of INVALID_SOCKET INVALID_SOCKET is a special value in Windows representing a non-valid socket identifier. We were #defining this to -1 on non-Windows platforms, causing unneccessary namespace pollution. Let's have our own identifier instead. Thanks to Matt Lawson for pointing this out. - nw/Makefile.netware: Fix project name typo to avoid needless confusion - example/x11: Set raw terminal mode manually instead of with cfmakeraw() OpenSolaris has no cfmakeraw() so to make the example more portable we simply do the equivalent operations on struct termios ourselves. Thanks to Tom Weber for reporting this problem, and finding a solution. Daniel Stenberg (17 Feb 2012) - sftp_write: cannot return acked data *and* EAGAIN Whenever we have acked data and is about to call a function that *MAY* return EAGAIN we must return the number now and wait to get called again. Our API only allows data *or* EAGAIN and we must never try to get both. Peter Stuge (13 Feb 2012) - example/x11: Build only when sys/un.h is found by configure The example can't be built on systems without AF_UNIX sockets. Daniel Stenberg (10 Feb 2012) - [Alexander Lamaison brought this change] Simplified sftp_read. Removed the total_read variable that originally must have tracked how much data had been written to the buffer. With non-blocking reads, we must return straight away once we have read data into the buffer so this variable served not purpose. I think it was still hanging around in case the initial processing of 'leftover' data meant we wrote to the buffer but this case, like the others, must return immediately. Now that it does, the last remaining need for the variable is gone. - [Alexander Lamaison brought this change] Cleaned up sftp_read and added more explanation. Replaced the gotos which were implementing the state machine with a switch statement which makes the states more explicit. - sftp_read: avoid data *and* EAGAIN Whenever we have data and is about to call a function that *MAY* return EAGAIN we must return the data now and wait to get called again. Our API only allows data *or* EAGAIN and we must never try to get both. Peter Stuge (2 Feb 2012) - Add a tcpip-forward example which demonstrates remote port forwarding - libssh2.h: Add missing prototype for libssh2_session_banner_set() - example/subsystem_netconf.c: Return error when read buffer is too small Also remove a little redundancy in the read loop condition. - example/subsystem_netconf.c: Add a missing newline in an error message - Fix undefined reference to _libssh_error in libgcrypt backend Commit 209de22299b4b58e582891dfba70f57e1e0492db introduced a function call to a non-existing function, and since then the libgcrypt backend has not been buildable. Version 1.4.0 (31 Jan 2012) Daniel Stenberg (31 Jan 2012) - RELEASE-NOTES: synced with 6bd584d29 for 1.4.0 - s/1.3.1/1.4.0 We're bumping the minor number - [Jernej Kovacic brought this change] libssh2_session_supported_algs: fix compiler warning - [Jernej Kovacic brought this change] session_supported_algs docs: added an example - [Gellule Xg brought this change] sftp-seek: clear EOF flag Set the EOF flag to False when calling seek64 to be able to get some data back on a following read - [Peter Krempa brought this change] userauth: Provide more informations if ssh pub key extraction fails If the function that extracts/computes the public key from a private key fails the errors it reports were masked by the function calling it. This patch modifies the key extraction function to return errors using _libssh_error() function. The error messages are tweaked to contain reference to the failed operaton in addition to the reason. * AUTHORS: - add my name * libgcrypt.c: _libssh2_pub_priv_keyfile(): - return a more verbose error using _libssh2_error() func. * openssl.c: - modify call graph of _libssh2_pub_priv_keyfile() to use _libssh2_error for error reporting(); * userauth.c: - tweak functions calling _libssh2_pub_priv_keyfile() not to shadow error messages - TODO: remove issues we (sort of) did already - ssh2_exec: skip error outputs for EAGAIN Since the example uses non-blocking mode, it will just flood the output with this "nonsense" error. Guenter Knauf (30 Nov 2011) - Some NetWare makefile tweaks. Daniel Stenberg (18 Nov 2011) - LIBSSH2_SFTP_PACKET_MAXLEN: increase to 80000 Some SFTP servers send SFTP packets larger than 40000. Since the limit is only present to avoid insane sizes anyway, we can easily bump it. The define was formerly in the public header libssh2_sftp.h but served no external purpose and was moved into the source dir. Bug: http://www.libssh2.org/mail/libssh2-devel-archive-2011-11/0004.shtml Reported by: Michael Harris Alexander Lamaison (18 Nov 2011) - [Peter Krempa brought this change] knownhost_check(): Don't dereference ext if NULL is passed Documentation for libssh2_knownhost_checkp() and related functions states that the last argument is filled with data if non-NULL. "knownhost if set to non-NULL, it must be a pointer to a 'struct libssh2_knownhost' pointer that gets filled in to point to info about a known host that matches or partially matches." In this function ext is dereferenced even if set to NULL, causing segfault in applications not needing the extra data. Daniel Stenberg (11 Nov 2011) - [Peter Krempa brought this change] knownhost_add: Avoid dereferencing uninitialized memory on error path. In function knownhost_add, memory is alocated for a new entry. If normal alocation is used, memory is not initialized to 0 right after, but a check is done to verify if correct key type is passed. This test is done BEFORE setting the memory to null, and on the error path function free_host() is called, that tries to dereference unititialized memory, resulting into a glibc abort(). * knownhost.c - knownhost_add(): - move typemask check before alloc - windows build: add define to avoid compiler warning A recent mingw compiler has started to complain on "#warning Please include winsock2.h before windows.h" unless the magic define is set first. Reported by: Vincent Torri Bug: http://www.libssh2.org/mail/libssh2-devel-archive-2011-10/0064.shtml Henrik Nordstrom (31 Oct 2011) - [Vincent Torri brought this change] Correct Windows include file name case, simplifying cross-compilation When cross compiling to Windows, libssh2.h include Windows header files with upper case filenames : BaseTsd.h and WinSock2.h. These files have lowercase names with mingw-w64 (iirc, it's the same with mingw). And as on Windows, being lowercase or uppercase does not matter. Daniel Stenberg (25 Oct 2011) - [Jernej Kovacic brought this change] libssh2_session_supported_algs: added - [Kamil Dudka brought this change] example/sftp_RW_nonblock: do not ignore LIBSSH2_ERROR_EAGAIN Bug: https://bugzilla.redhat.com/745420 Peter Stuge (5 Oct 2011) - example/ssh2_agent: Print host key fingerprint before authentication Also moves the comment about not being authenticated to before the agent authentication takes place, so that it better matches the code. Daniel Stenberg (29 Sep 2011) - OpenSSL EVP: fix threaded use of structs Make sure we don't clear or reset static structs after first init so that they work fine even when used from multiple threads. Init the structs in the global init. Help and assistance by: John Engstrom Fixes #229 (again) - openssl: don't init static structs differently make_ctr_evp() is changed to take a struct pointer, and then each _libssh2_EVP_aes_[keylen]_ctr function is made to pass in their own static struct Reported by: John Engstrom Fixes #229 Guenter Knauf (27 Sep 2011) - Removed obsolete include path. Daniel Stenberg (21 Sep 2011) - read_state: clear the state variable better Set read_state back to idle before trying to send anything so that if the state somehow is wrongly set. Also, avoid such a case of confusion by resetting the read_state when an sftp handle is closed. - sftp_read: remove leftover fprintf Reported by: Alexander Lamaison - sftp.h: fix the #ifdef to prevent multiple inclusions - sftp_read: use a state variable to avoid bad writes When a channel_write call has gotten an EAGAIN back, we try harder to continue the same write in the subsequent invoke. - window_size: explicit adjustments only Removed the automatic window_size adjustments from _libssh2_channel_read() and instead all channel readers must now make sure to enlarge the window sizes properly themselves. libssh2_channel_read_ex() - the public function, now grows the window size according to the requested buffer size. Applications can still opt to grow the window more on demand. Larger windows tend to give higher performance. sftp_read() now uses the read-ahead logic to figure out a window_size. - libssh2.h: bump the default window size to 256K - libssh2_userauth_keyboard_interactive.3: fix man warning It seemed to occur due to the excessive line length - [Mikhail Gusarov brought this change] Add missing .gitignore entries - [Mikhail Gusarov brought this change] Add manpage syntax checker to 'check' target In virtually every libssh2 release Debian's lintian catches syntax errors in manpages. Prevent it by checking manpages as a part of testsuite. - libssh2_banner_set.3: fix nroff syntax mistake Guenter Knauf (10 Sep 2011) - Use predefined resource compiler macro. - Added casts to silent compiler warnings. - Fixed uint64_t printf. - Fixed macro function signatures. - NetWare makefile tweaks. - Removed unused var. - Added 2 samples not mentioned. - Dont build x11 sample with MinGW. - Fixed executable file description. - Removed unused var. - Kill stupid gcc 3.x uninitialized warning. - Build all examples. - More MinGW makefile tweaks. Renamed *.mingw makefiles to GNUmakefile since GNU make picks these up automatically, and therefore win32/Makefile removed. - Removed forgotten WINSOCK_VERSION defines. Daniel Stenberg (9 Sep 2011) - libssh2_session_startup(3) => libssh2_session_handshake(3) Propagate for the current function in docs and examples. libssh2_session_startup() is deprecated. - libssh2_banner_set => libssh2_session_banner_get Marked the old function as deprecated. Added the new name in the correct name space with the same arguments and functionality. - new function: libssh2_session_banner_get Returns the banner from the server handshake Fixes #226 - libssh2.h: bump version to 1.4.0 for new function(s) - remove embedded CVS/svn tags - [liuzl brought this change] API add:libssh2_sftp_get_channel Return the channel of sftp, then caller can control the channel's behavior. Signed-off-by: liuzl <xieepp@gmail.com> - _libssh2_channel_read: react on errors from receive_window_adjust Previously the function would ignore all errors except for EAGAIN. - sftp_read: extend and clarify the documentation - sftp_read: cap the read ahead maximum amount Now we only go up to LIBSSH2_CHANNEL_WINDOW_DEFAULT*30 bytes SFTP read ahead, which currently equals 64K*30 == 1966080 bytes. - _libssh2_channel_read: fix non-blocking window adjusting If EAGAIN is returned when adjusting the receive window, we must not read from the transport directly until we've finished the adjusting. Guenter Knauf (8 Sep 2011) - Fix for systems which need sys/select.h. - The files were not gone but renamed ... Daniel Stenberg (6 Sep 2011) - sftp_read: added documenting comment Taken from some recent email conversations I added some descriptions of the logic in sftp_read() to aid readers. - 1.3.1: start the work Version 1.3.0 (6 Sep 2011) Daniel Stenberg (6 Sep 2011) - Makefile.am: the Makefile.win32 files are gone - RELEASE-NOTES: updated for 1.3.0 - sftp_read: a short read is not end of file A returned READ packet that is short will now only reduce the offset. This is a temporary fix as it is slightly better than the previous approach but still not very good. - [liuzl brought this change] _libssh2_packet_add: adjust window size when truncating When receiving more data than what the window size allows on a particular channel, make sure that the window size is adjusted in that case too. Previously it would only adjust the window in the non-error case. Guenter Knauf (29 Aug 2011) - Silent compiler warning with MinGW64. - Fixed link to native Win32 awk tool. - Renamed MinGW makefiles. - Some MinGW makefile tweaks. Enable build without GNU tools and with MinGW64 compiler. - Fixed aes_ctr_do_cipher() signature. Daniel Stenberg (26 Aug 2011) - [liuzl brought this change] libssh2_sftp_seek64: flush packetlist and buffered data When seeking to a new position, flush the packetlist and buffered data to prevent already received or pending data to wrongly get used when sftp-reading from the new offset within the file. - sftp_read: advance offset correctly for buffered copies In the case where a read packet has been received from the server, but the entire contents couldn't be copied to the user-buffer, the data is instead buffered and copied to the user's buffer in the next invocation of sftp_read(). When that "extra" copy is made, the 'offset' pointer was not advanced accordingly. The biggest impact of this flaw was that the 'already' variable at the top of the function that figures out how much data "ahead" that has already been asked for would slowly go more and more out of sync, which could lead to the file not being read all the way to the end. This problem was most noticable in cases where the application would only try to read the exact file size amount, like curl does. In the examples libssh2 provides the sftp read function is most often called with a fixed size large buffer and then the bug would not appear as easily. This bug was introduced in the SFTP rewrite in 1.2.8. Bug: http://curl.haxx.se/mail/lib-2011-08/0305.html http://www.libssh2.org/mail/libssh2-devel-archive-2011-08/0085.shtml - wrap some long lines < 80 columns - LIBSSH2_RECV: fix typo, use the RECV_FD macro - subsystem_netconf.c: fix compiler warnings - [Henrik Nordstrom brought this change] Custom callbacks for performing low level socket I/O - version bump: start working towards 1.3.0 Version 1.2.9 (16 Aug 2011) Daniel Stenberg (16 Aug 2011) - RELEASE-NOTES: synced with 95d69d3a81261 - [Henrik Nordstrom brought this change] Document prototypes for macro defined functions - [Henrik Nordstrom brought this change] Avoid reuse after free when closing X11 channels - _libssh2_channel_write: handle window_size == 0 better When about to send data on the channel and the window size is 0, we must not just return 0 if the transport_read() function returned EAGAIN as it then causes a busy-loop. Bug: http://libssh2.org/mail/libssh2-devel-archive-2011-08/0011.shtml - gettimeofday: fix name space pollution For systems without its own gettimeofday() implementation, we still must not provide one outside our namespace. Reported by: Bill Segall Dan Fandrich (5 Aug 2011) - libssh2.pc.in: Fixed spelling in pkgconfig file Peter Stuge (17 Jul 2011) - example/subsystem_netconf.c: Add missing #include <string.h> - example/subsystem_netconf.c: Discard ]]>]]> and return only XML response - example/subsystem_netconf.c: Fix uninitialized variable bug - example: Add subsystem_netconf.c This example demonstrates how to use libssh2 to send a request to the NETCONF subsystem available e.g. in JunOS. See also http://tools.ietf.org/html/draft-ietf-netconf-ssh-06 Daniel Stenberg (16 Jul 2011) - man page cleanups: non-existing functions need no man pages - libssh2_new_host_entry.3: removed This is just junk leftovers. - userauth_keyboard_interactive: fix buffer overflow Partly reverse 566894494b4972ae12 which was simplifying the code far too much and ended up overflowing a buffer within the LIBSSH2_SESSION struct. Back to allocating the buffer properly like it used to do. Bug: http://www.libssh2.org/mail/libssh2-devel-archive-2011-06/0032.shtml Reported by: Alfred Gebert - keyboard-interactive man page: cleaned up - [Alfred Gebert brought this change] _libssh2_recv(): handle ENOENT error as EAGAIN A sftp session failed with error "failure establishing ssh session" on Solaris and HP-UX. Sometimes the first recv() function call sets errno to ENOENT. In the man pages for recv of Solaris and HP-UX the error ENOENT is not documented. I tested Solaris SPARC and x86, HP-UX i64, AIX, Windows and Linux. - agent_list_identities: fix out of scope access An auto variable out of scope was being referenced and used. fixes #220 - _libssh2_wait_socket: fix timeouts for poll() uses - windows: inclusion fix include winsock2.h for all windows compilers - keyb-interactive: add the fixed buffer Belongs to commit 5668944 - code cleanup: don't use C99/c++ comments We aim for C89 compliance - keyb-interactive: allow zero length fields Allow zero length fields so they don't cause malloc(0) calls Avoid free()ing NULL pointers Avoid a malloc of a fixed 5 byte buffer. - libssh2_channel_process_startup.3: clean up Remove the references to the macro-fied shortcuts as they have their own individual man pages. Made the prototype different and more readable. - man page: fix .BR lines We don't use \fI etc on .BR lines - userauth_keyboard_interactive: skip code on zero length auth - libssh2_channel_forward_accept.3: mention how to get error Since this returns a pointer, libssh2_session_last_errno() must be used to get the actual error code and it wasn't that clear before. - timeout docs: mention they're added in 1.2.9 - sftp_write_sliding.c: indent fix Use the standard indenting and removed CVS leftover comment - [zl liu brought this change] sftp_write_sliding: send the complete file When reaching the end of file there can still be data left not sent. - [Douglas Masterson brought this change] session_startup: init state properly libssh2_session_startup() didn't set the state correctly so it could get confused. Fixes #218 - timeout: added man pages - BLOCK_ADJUST_ERRNO: move rc to right level We can't declare the variable within the block and use it in the final do-while() expression to be properly portable C89. - [Matt Lilley brought this change] adds a timeout to blocking calls Fixes bug #160 as per Daniel's suggestion Adds libssh2_session_set_timeout() and libssh2_session_get_timeout() - SCP: fix incorrect error code After an error occurs in libssh2_scp_recv() or libssh2_scp_send(), the function libssh2_session_last_error() would return LIBSSH2_ERROR_SOCKET_NONE on error. Bug: http://trac.libssh2.org/ticket/216 Patch by: "littlesavage" Fixes #216 Guenter Knauf (19 Apr 2011) - Updated default (recommended) dependency versions. Daniel Stenberg (17 Apr 2011) - libssh2_session_block_directions: fix mistake The last LIBSSH2_SESSION_BLOCK_INBOUND should be LIBSSH2_SESSION_BLOCK_OUTBOUND And I shortened the short description Reported by: "drswinghead" - msvcproj: added libs and debug stuff Added libraries needed to link whether using openssl dynamically or statically Added LIBSSH2DEBUG define to debug versions to enable tracing URL: http://trac.libssh2.org/ticket/215 Patch by: Mark Smith - sftp_write: clean offsets on error When an error has occurred on FXP_WRITE, we must make sure that the offset, sent offset and acked counter are reset properly. - example/.gitignore: ignore built binaries - sftp_write: flush the packetlist on error When an error occurs during write, flush the entire list of pending outgoing SFTP packets. - keepalive: add first basic man pages Someone on IRC pointed out that we don't have these documented so I wrote up a first set based on the information in the wiki: http://trac.libssh2.org/wiki/KeepAlive - scp_write_nonblock.c: remove pointless check libssh2_channel_write() cannot return a value that is larger than the input length value Mikhail Gusarov (9 Apr 2011) - s/\.NF/.nf/ to fix wrong macro name caught by man --warnings Daniel Stenberg (6 Apr 2011) - version: bump to 1.2.9_dev Also update the copyright year range to include 2011 - configure: fix $VERSION Stop using the $VERSION variable as it seems to be magically used by autoconfig itself and thus gets set to the value set in AC_INIT() without us wanting that. $LIBSSH2VER is now the libssh2 version as detected. Reported by: Paul Howarth Bug: http://www.libssh2.org/mail/libssh2-devel-archive-2011-04/0008.shtml - maketgz: use git2news.pl by the correct name Version 1.2.8 (4 Apr 2011) Daniel Stenberg (4 Apr 2011) - RELEASE-NOTES: synced with fabf1a45ee - NEWS: auto-generated from git Starting now, the NEWS file is generated from git using the git2news.pl script. This makes it always accurate and up-to-date, even for daily snapshots etc. - sftp_write: handle FXP_WRITE errors When an sftp server returns an error back on write, make sure the function bails out and returns the proper error. - configure: stop using the deprecated AM_INIT_AUTOMAKE syntax Alexander Lamaison (13 Mar 2011) - Support unlimited number of host names in a single line of the known_hosts file. Previously the code assumed either a single host name or a hostname,ip-address pair. However, according to the spec [1], there can be any number of comma separated host names or IP addresses. [1] http://www.openbsd.org/cgi-bin/man.cgi?query=sshd&sektion=8 Daniel Stenberg (26 Feb 2011) - libssh2_knownhost_readfile.3: clarify return value This function returns the number of parsed hosts on success, not just zero as previously documented. Peter Stuge (26 Feb 2011) - Don't save allocated packet size until it has actually been allocated The allocated packet size is internal state which needs to match reality in order to avoid problems. This commit fixes #211. Daniel Stenberg (21 Feb 2011) - [Alfred Gebert brought this change] session_startup: manage server data before server identification Fix the bug that libssh2 could not connect if the sftp server sends data before sending the version string. http://tools.ietf.org/html/rfc4253#section-4.2 "The server MAY send other lines of data before sending the version string. Each line SHOULD be terminated by a Carriage Return and Line Feed. Such lines MUST NOT begin with "SSH-", and SHOULD be encoded in ISO-10646 UTF-8 [RFC3629] (language is not specified). Clients MUST be able to process such lines." - [Alfred Gebert brought this change] fullpacket: decompression only after init The buffer for the decompression (remote.comp_abstract) is initialised in time when it is needed. With this fix decompression is disabled when the buffer (remote.comp_abstract) is not initialised. Bug: http://trac.libssh2.org/ticket/200 - _libssh2_channel_read: store last error When the transport layer returns EAGAIN this function didn't call _libssh2_error() which made the last_error not get set. - sftp_write: clarified the comment header - sftp_read: avoid wrapping counter to insanity As pointed out in bug #206, if a second invoke of libssh2_sftp_read() would shrink the buffer size, libssh2 would go nuts and send out read requests like crazy. This was due to an unsigned variable turning "negative" by some wrong math, and that value would be the amount of data attempt to pre-buffer! Bug: http://trac.libssh2.org/ticket/206 - sftp_packet_read: use 32bit variables for 32bit data - libssh2_sftp_stat_ex.3: cleaned up, extended Removed the macros from it as they have their own man pages. Added the LIBSSH2_SFTP_ATTRIBUTES struct in here for easier reference. - sftp_readdir: return error if buffer is too small If asked to read data into a buffer and the buffer is too small to hold the data, this function now returns an error instead of as previously just copy as much as fits. - sftp_symlink: return error if receive buffer too small and clean up some variable type mismatches Discussion: http://www.libssh2.org/mail/libssh2-devel-archive-2011-01/0001.shtml - docs: clarify what happens with a too small buffer This flaw is subject to change, but I figured it might be valuable to users of existing code to know how it works. - channel_request_pty_size: fix reqPTY_state The state variable isn't properly set so every other call to the function fails! Bug: http://libssh2.org/mail/libssh2-devel-archive-2010-12/0096.shtml Reported by: Steve Legg - data size: cleanup Fix 64bit warnings by using (s)size_t and dedicated uint32_t types more. - [Pierre Joye brought this change] ssize_t: proper typedef with MSVC compilers As discussed on the mailing list, it was wrong for win64 and using the VC-provided type is the safest approach instead of second- guessing which one it should be. Guenter Knauf (22 Dec 2010) - Updated OpenSSL version. - Expanded tabs to spaces. Peter Stuge (21 Dec 2010) - [Joey Degges brought this change] _libssh2_ntohu64: fix conversion from network bytes to uint64 Cast individual bytes to uint64 to avoid overflow in arithmetic. Daniel Stenberg (20 Dec 2010) - libssh2_userauth_list: language fix "faily" is not a good English word, and I also cleaned up some other minor mistakes - crypto: unify the generic functions Added crypto.h that is the unified header to include when using crypto functionality. It should be the only header that needs to adapt to the underlying crypto library in use. It provides the set of prototypes that are library agnostic. - [Mark Smith brought this change] userauth: derive publickey from private Pass a NULL pointer for the publickey parameter of libssh2_userauth_publickey_fromfile and libssh2_userauth_hostbased_fromfile functions. In this case, the functions recompute the public key from the private key file data. This is work done by Jean-Louis CHARTON <Jean-Louis.CHARTON@oikialog.com>, then adapted by Mark Smith and slightly edited further by me Daniel. WARNING: this does leave the feature NOT WORKING when libssh2 is built to use libgcrypt instead of OpenSSL simply due to lack of implementation. - ssh2_echo: Value stored to 'exitcode' is never read - _libssh2_packet_add: fix SSH_MSG_DEBUG weirdness I believe I may have caused this weird typo style error when I cleaned up this function a while ago. Corrected now. - uint32: more longs converted to proper types I also moved the MAC struct over to the mac.h header file and made sure that the users of that struct include that file. - SFTP: more types to uint32_t The 'num_names' field in the SSH_FXP_NAME response is an unsigned 32bit value so we make sure to treat it like that. - SFTP: request_ids are uint32_t I went over the code and made sure we use uint32_t all over for the request_id data. It is an unsigned 32bit value on the wire. - SFTP: store request_id separately in packets By using a new separate struct for incoming SFTP packets and not sharing the generic packet struct, we can get rid of an unused field and add a new one dedicated for holding the request_id for the incoming package. As sftp_packet_ask() is called fairly often, a "mere" integer comparison is MUCH faster than the previous memcmp() of (typically) 5 bytes. - libssh2_sftp_open_ex: man page extended and cleaned up I added the missing documentation for the 'flags' argument. - SFTP: unify the READ/WRITE chunk structs - SFTP: fix memory leaks Make sure that we cleanup remainders when the handle is closed and when the subsystem is shutdown. Existing flaw: if a single handle sends packets that haven't been replied to yet at the time when the handle is closed, those packets will arrive later and end up in the generic packet brigade queue and they will remain in there until flushed. They will use unnecessary memory, make things slower and they will ruin the SFTP handling if the request_id counter ever wraps (highly unlikely to every happen). - sftp_close_handle: packet list is generic Fix comment, simplify the loop logic - sftp_read: pipeline reads The SFTP read function now does transfers the same way the SFTP write function was made to recently: it creates a list of many outgoing FXP_READ packets that each asks for a small data chunk. The code then tries to keep sending read request while collecting the acks for the previous requests and returns the received data. - sftp_write: removed unused variable - _libssh2_channel_close: don't call transport read if disconnected The loop that waits for remote.close to get set may end up looping forever since session->socket_state gets set to LIBSSH2_SOCKET_DISCONNECTED by the packet_add() function called from the transport_read() function and after having been set to LIBSSH2_SOCKET_DISCONNECTED, the transport_read() function will only return 0. Bug: http://trac.libssh2.org/ticket/198 - libssh2_sftp_seek64: new man page Split off libssh2_sftp_seek64 from the libssh2_sftp_seek man page, and mentioned that we consider the latter deprecated. Also added a mention about the dangers of doing seek during writing or reading. - sftp_seek: fix The new SFTP write code caused a regression as the seek function no longer worked as it didn't set the write position properly. It should be noted that seeking is STRONGLY PROHIBITED during upload, as the upload magic uses two different offset positions and the multiple outstanding packets etc make them sensitive to change in the midst of operations. This functionality was just verified with the new example code sftp_append. This bug was filed as bug #202: Bug: http://trac.libssh2.org/ticket/202 - sftp_append: new example doing SFTP append - MAX_SFTP_OUTGOING_SIZE: 30000 I ran SFTP upload tests against localhost. It showed that to make the app reach really good speeds, I needed to do a little code tweak and change MAX_SFTP_OUTGOING_SIZE from 4000 to 30000. The tests I did before with the high latency tests didn't show any real difference whatever I had that size set to. This number is the size in bytes that libssh2 cuts off the large input buffer and sends off as an individual sftp packet. - sftp_write_sliding.c: new example This is an example that is very similar to sftp_write_nonblock.c, with the exception that this uses 1 - a larger upload buffer 2 - a sliding buffer mechnism to allow the app to keep sending lots of data to libssh2 without having to first drain the buffer. These are two key issues to make libssh2 SFTP uploads really perform well at this point in time. - cpp: s/#elsif/#elif This looks like a typo as #elsif is not really C... - _libssh2_channel_write: revert channel_write() use The attempts made to have _libssh2_channel_write() accept larger pieces of data and split up the data by itself into 32700 byte chunks and pass them on to channel_write() in a loop as a way to do faster operations on larger data blocks was a failed attempt. The reason why it is difficult: The API only allows EAGAIN or a length to be returned. When looping over multiple blocks to get sent, one block can get sent and the next might not. And yet: when transport_send() has returned EAGAIN we must not call it again with new data until it has returned OK on the existing data it is still working on. This makes it a mess and we do get a much easier job by simply returning the bytes or EAGAIN at once, as in the EAGAIN case we can assume that we will be called with the same arguments again and transport_send() will be happy. Unfortunately, I think we take a small performance hit by not being able to do this. - ssh2_echo: new example This is a new example snippet. The code is largely based on ssh2_exec, and is written by Tommy Lindgren. I edited it into C90 compliance and to conform to libssh2 indent style and some more. - send_existing: return after send_existing When a piece of data is sent from the send_existing() function we must make the parent function return afterwards. Otherwise we risk that the parent function tries to send more data and ends up getting an EGAIN for that more data and since it can only return one return code it doesn't return info for the successfully sent data. As this change is a regression I now added a larger comment explaining why it has to work like this. - _libssh2_channel_write: count resent data as written In the logic that resends data that was kept for that purpose due to a previous EAGAIN, the data was not counted as sent causing badness. Peter Stuge (13 Nov 2010) - Use fprintf(stderr, ) instead of write(2, ) for debugging - session/transport: Correctly handle when _libssh2_send() returns -EAGAIN - src/agent.c: Simplify _libssh2_send() error checking ever so slightly Daniel Stenberg (12 Nov 2010) - send/recv: use _libssh2_recv and _libssh2_send now Starting now, we unconditionally use the internal replacement functions for send() and recv() - creatively named _libssh2_recv() and _libssh2_send(). On errors, these functions return the negative 'errno' value instead of the traditional -1. This design allows systems that have no "natural" errno support to not have to invent it. It also means that no code outside of these two transfer functions should use the errno variable. - channel_write: move some logic to _libssh2_channel_write Some checks are better done in _libssh2_channel_write just once per write instead of in channel_write() since the looping will call the latter function multiple times per _libssh2_channel_write() invoke. - sftp_write: handle "left over" acked data The SFTP handle struct now buffers number of acked bytes that haven't yet been returned. The way this is used is as following: 1. sftp_write() gets called with a buffer of let say size 32000. We split 32000 into 8 smaller packets and send them off one by one. One of them gets acked before the function returns so 4000 is returned. 2. sftp_write() gets called again a short while after the previous one, now with a much smaller size passed in to the function. Lets say 8000. In the mean-time, all of the remaining packets from the previous call have been acked (7*4000 = 28000). This function then returns 8000 as all data passed in are already sent and it can't return any more than what it got passed in. But we have 28000 bytes acked. We now store the remaining 20000 in the handle->u.file.acked struct field to add up in the next call. 3. sftp_write() gets called again, and now there's a backlogged 20000 bytes to return as fine and that will get skipped from the beginning of the buffer that is passed in. - sftp_write: polished and simplified Removed unnecessary struct fields and state changes within the function. Made the loop that checks for ACKs only check chunks that were fully sent. - SCP: on failure, show the numerical error reason By calling libssh2_session_last_errno() - SFTP: provide the numerical error reason on failure - SCP: clean up failure treatment When SCP send or recv fails, it gets a special message from the server with a warning or error message included. We have no current API to expose that message but the foundation is there. Removed unnecessary use of session struct fields. - sftp_write: enlarge buffer to perform better - packets: code cleanup I added size checks in several places. I fixed the code flow to be easier to read in some places. I removed unnecessary zeroing of structs. I removed unused struct fields. - LIBSSH2_CALLBACK_MACERROR: clarify return code use - _libssh2_userauth_publickey: avoid shadowing - packet: avoid shadowing global symbols - sftp_readdir: avoid shadowing - shadowing: don't shadow the global compress - _libssh2_packet_add: turn ifs into a single switch - _libssh2_packet_add: check SSH_MSG_GLOBAL_REQUEST packet - _libssh2_packet_add: SSH_MSG_DEBUG length checks Verify lengths before using them. Read always_display from the correct index. Don't copy stuff around just to provide zero-termination of the strings. - _libssh2_packet_add: SSH_MSG_IGNORE skip memmove There's no promise of a zero termination of the data in the callback so no longer perform ugly operation in order to provide it. - _libssh2_packet_add: SSH_MSG_DISCONNECT length checks Verify lengths before trying to read data. - indent: break lines at 80 columns - SSH_MSG_CHANNEL_OPEN_FAILURE: used defined values We don't like magic numbers in the code. Now the acceptable failure codes sent in the SSH_MSG_CHANNEL_OPEN_FAILURE message are added as defined values in the private header file. - sftp_write: don't return EAGAIN if no EAGAIN was received This function now only returns EAGAIN if a lower layer actually returned EAGAIN to it. If nothing was acked and no EAGAIN was received, it will now instead return 0. - _libssh2_wait_socket: detect nothing-to-wait-for If _libssh2_wait_socket() gets called but there's no direction set to wait for, this causes a "hang". This code now detects this situation, set a 1 second timeout instead and outputs a debug output about it. - decomp: remove the free_dest argument Since the decompress function ALWAYS returns allocated memory we get a lot simpler code by removing the ability to return data unallocated. - decomp: cleaned off old compression stuff I cleared off legacy code from when the compression and decompression functions were a single unified function. Makes the code easier to read too. - [TJ Saunders brought this change] decomp: increase decompression buffer sizes - [TJ Saunders brought this change] zlib: Add debug tracing of zlib errors - sftp_packet_read: handle partial reads of the length field SFTP packets come as [32 bit length][payload] and the code didn't previously handle that the initial 32 bit field was read only partially when it was read. - [Jasmeet Bagga brought this change] kex_agree_hostkey: fix NULL pointer derefence While setting up the session, ssh tries to determine the type of encryption method it can use for the session. This requires looking at the keys offered by the remote host and comparing these with the methods supported by libssh2 (rsa & dss). To do this there is an iteration over the array containing the methods supported by libssh2. If there is no agreement on the type of encryption we come to the 3rd entry of the hostkeyp array. Here hostkeyp is valid but *hostkep is NULL. Thus when we dereference that in (*hostkeyp)->name there is a crash - _libssh2_transport_send: remove dead assignment 'data' isn't accessed beyond this point so there's no need to assign it. - scp_recv: remove dead assignment Instead of assigning a variable we won't read, we now use the more explicit (void) prefix. - sftp_write: removed superfluous assignment - bugfix: avoid use of uninitialized value - sftp_packet_require: propagate error codes better There were some chances that they would cause -1 to get returned by public functions and as we're hunting down all such occurances and since the underlying functions do return valuable information the code now passes back proper return codes better. - [Alfred Gebert brought this change] fix memory leaks (two times cipher_data) for each sftp session - libssh2_userauth_authenticated: make it work as documented The man page clearly says it returns 1 for "already authenticated" but the code said non-zero. I changed the code to use 1 now, as that is also non-zero but it gets the benefit that it now matches the documentation. Using 1 instead of non-zero is better for two reasons: 1. We have the opportunity to introduce other return codes in the future for things like error and what not. 2. We don't expose the internal bitmask variable value. - userauth_keyboard_interactive: fix indent - [Alfred Gebert brought this change] fix memory leak in userauth_keyboard_interactive() First I wanted to free the memory in session_free() but then I had still memory leaks because in my test case the function userauth_keyboard_interactive() is called twice. It is called twice perhaps because the server has this authentication methods available: publickey,gssapi-with-mic,keyboard-interactive The keyboard-interactive method is successful. - dist: include sftp.h in dist archives Simon Josefsson (27 Oct 2010) - Update header to match new function prototype, see c48840ba88. Daniel Stenberg (26 Oct 2010) - bugfixes: the transport rearrange left some subtle flaws now gone - libssh2_userauth_publickey_fromfile_ex.3: cleaned up looks - libssh2_userauth_publickey: add man page I found an undocumented public function and we can't have it like that. The description here is incomplete, but should serve as a template to allow filling in... - libssh2_sftp_write.3: added blurb about the "write ahead" Documented the new SFTP write concept - sftp_close_handle: free any trailing write chunks - _libssh2_channel_write: fix warnings - SFTP: bufgix, move more sftp stuff to sftp.h The sftp_write function shouldn't assume that the buffer pointer will be the same in subsequent calls, even if it assumes that the data already passed in before haven't changed. The sftp structs are now moved to sftp.h (which I forgot to add before) - SFTP: use multiple outgoing packets when writing sftp_write was rewritten to split up outgoing data into multiple packets and deal with the acks in a more asynchronous manner. This is meant to help overcome latency and round-trip problems with the SFTP protocol. - TODO: implemented a lot of the ideas now - _libssh2_channel_write: removed 32500 size limit Neither _libssh2_channel_write nor sftp_write now have the 32500 size limit anymore and instead the channel writing function now has its own logic to send data in multiple calls until everything is sent. - send_existing: don't tell parent to return when drained That will just cause unnecessary code execution. - _libssh2_channel_write: general code cleanup simplified the function and removed some unused struct fields - _libssh2_transport_send: replaces _libssh2_transport_write The new function takes two data areas, combines them and sends them as a single SSH packet. This allows several functions to allocate and copy less data. I also found and fixed a mixed up use of the compression function arguments that I introduced in my rewrite in a recent commit. - scp_write_nonblock: use select() instead of busyloop Make this example nicer by not busylooping. - send_existing: clear olen when the data is sent off - _libssh2_transport_write: allow 256 extra bytes around the packet - _libssh2_transport_write: remade to send without malloc - compress: compression disabled by default We now allow libssh2_session_flag() to enable compression with a new flag and I added documentation for the previous LIBSSH2_FLAG_SIGPIPE flag which I wasn't really aware of! - comp: split the compress function It is now made into two separate compress and decompress functions. In preparation for upcoming further modficications. Dan Fandrich (20 Oct 2010) - Added header file to allow compiling in older environments Daniel Stenberg (20 Oct 2010) - TODO: add a possible new API for SFTP transfers - TODO: "New Transport API" added - TODO: add buffering plans Simon Josefsson (13 Oct 2010) - Mention libssh2_channel_get_exit_signal and give kudos. - [Tommy Lindgren brought this change] Add libssh2_channel_get_exit_signal man page. Signed-off-by: Simon Josefsson <simon@josefsson.org> - [Tommy Lindgren brought this change] Add libssh2_channel_get_exit_signal. Signed-off-by: Simon Josefsson <simon@josefsson.org> - Add libssh2_free man page and fix typo. - Add libssh2_free. Daniel Stenberg (11 Oct 2010) - scp_recv: improved treatment of channel_read() returning zero As a zero return code from channel_read() is not an error we must make sure that the SCP functions deal with that properly. channel_read() always returns 0 if the channel is EOFed already so we check for EOF after 0-reads to be able to return error properly. - libssh2_session_methods.3: detail what can be asked for - compression: send zlib before none As the list of algorithms in a preferred order we should send zlib before none to increase the chances that the server will let us do compression. - compress: faster check, better return codes In the transport functions we avoid a strcmp() now and just check a boolean instead. The compress/decompress function's return code is now acknowledged and used as actual return code in case of failures. - libssh2_session_handshake: replaces libssh2_session_startup() The function libssh2_session_startup() is now considered deprecated due to the portability issue with the socket argument. libssh2_session_handshake() is the name of the replacement. - libssh2_socket_t: now externally visible In preparation for upcominig changes, the libssh2_socket_t type is now typedef'ed in the public header. - _libssh2_transport_drain: removed This function proved not to be used nor useful. - _libssh2_channel_write: don't iterate over transport writes When a call to _libssh2_transport_write() succeeds, we must return from _libssh2_channel_write() to allow the caller to provide the next chunk of data. We cannot move on to send the next piece of data that may already have been provided in this same function call, as we risk getting EAGAIN for that and we can't return information both about sent data as well as EAGAIN. So, by returning short now, the caller will call this function again with new data to send. - _libssh2_transport_write: updated documentation blurb - _libssh2_transport_write: remove fprintf remainder Mistake from previous debugging - session: improved errors Replaced -1/SOCKET_NONE errors with appropriate error defines instead. Made the verbose trace output during banner receiving less annoying for non-blocking sessions. - crypt_init: use correct error define - _libssh2_error: hide EAGAIN for non-blocking sessions In an attempt to make the trace output less cluttered for non-blocking sessions the error function now avoids calling the debug function if the error is the EAGAIN and the session is non-blocking. - agent: use better error defines - comp_method_zlib_init: use correct error defines - transport: better error codes LIBSSH2_SOCKET_NONE (-1) should no longer be used as error code as it is (too) generic and we should instead use specific and dedicated error codes to better describe the error. - channel: return code and _libssh2_error cleanup Made sure that all transport_write() call failures get _libssh2_error called. - _libssh2_channel_write: limit to 32700 bytes The well known and used ssh server Dropbear has a maximum SSH packet length at 32768 by default. Since the libssh2 design current have a fixed one-to-one mapping from channel_write() to the packet size created by transport_write() the previous limit of 32768 in the channel layer caused the transport layer to create larger packets than 32768 at times which Dropbear rejected forcibly (by closing the connection). The long term fix is of course to remove the hard relation between the outgoing SSH packet size and what the input length argument is in the transport_write() function call. - libssh.h: add more dedicated error codes - SCP: allow file names with bytes > 126 When parsing the SCP protocol and verifying that the data looks like a valid file name, byte values over 126 must not be consider illegal since UTF-8 file names will use such codes. Reported by: Uli Zappe Bug: http://www.libssh2.org/mail/libssh2-devel-archive-2010-08/0112.shtml Dan Fandrich (25 Aug 2010) - Document the three sftp stat constants Guenter Knauf (18 Aug 2010) - Fixed Win32 makefile which was now broken at resource build. - It is sufficient to pipe stderr to NUL to get rid of the nasty messages. - [Author: Guenter Knauf brought this change] Removed Win32 ifdef completely for sys/uio.h. No idea why we had this ifdef at all but MSVC, MingW32, Watcom and Borland all have no sys/uio.h header; so if there's another Win32 compiler which needs it then it should be added explicitely instead of this negative list. - New files should also be added to Makefile.am. Otherwise they will never be included with release and snapshot tarballs ... Daniel Stenberg (18 Aug 2010) - version: bump to 1.2.8_DEV Version 1.2.7 (17 Aug 2010) Daniel Stenberg (17 Aug 2010) - release: updated to hold 1.2.7 info Guenter Knauf (17 Aug 2010) - Use the new libssh2.rc file. - Added resource file for libssh2.dll (shamelessly stolen from libcurl). - Updated Win32 MSVC dependencies versions. - Added include for sys/select.h to get fd.set on some platforms. - Added Watcom makefile borrowed from libcurl. This makefile compiles already all files fine for static lib, but needs final touch when I have OpenSSL fully working with shared libs and Watcom. - Added copyright define to libssh2.h and use it for binary builds. - Moved version defines up in order to include from .rc file. Blocked rest of header with ifndef so its possible to let the rc compiler only use the version defines. - Some minor makefile tweaks. Daniel Stenberg (2 Aug 2010) - example: treat the libssh2_channel_read() return code properly A short read is not an error. Only negative values are errors! - libssh2_wait_socket: reset error code to "leak" EAGAIN less Since libssh2 often sets LIBSSH2_ERROR_EAGAIN internally before _libssh2_wait_socket is called, we can decrease some amount of confusion in user programs by resetting the error code in this function to reduce the risk of EAGAIN being stored as error when a blocking function returns. - _libssh2_wait_socket: poll needs milliseconds As reported on the mailing list, the code path using poll() should multiple seconds with 1000 to get milliseconds, not divide! Reported by: Jan Van Boghout - typedef: make ssize_t get typedef without LIBSSH2_WIN32 The condition around the ssize_t typedef depended on both LIBSSH2_WIN32 *and* _MSC_VER being defined when it should be enough to depend on _MSC_VER only. It also makes it nicer so libssh2-using code builds fine without having custom defines. - [John Little brought this change] session_free: free more data to avoid memory leaks - channel_free: ignore problems with channel_close() As was pointed out in bug #182, we must not return failure from _libssh2_channel_free() when _libssh2_channel_close() returns an error that isn't EAGAIN. It can effectively cause the function to never go through, like it did now in the case where the socket was actually closed but socket_state still said LIBSSH2_SOCKET_CONNECTED. I consider this fix the right thing as it now also survives other errors, even if making sure socket_state isn't lying is also a good idea. - publickey_list_free: no return value from a void function Fixed a compiler warning I introduced previously when checking input arguments more. I also added a check for the other pointer to avoid NULL pointer dereferences. - [Lars Nordin brought this change] openssl: make use of the EVP interface Make use of the EVP interface for the AES-funktion. Using this method supports the use of different ENGINES in OpenSSL for the AES function (and the direct call to the AES_encrypt should not be used according to openssl.org) Peter Stuge (23 Jun 2010) - [Tor Arntsen brought this change] Don't overflow MD5 server hostkey Use SHA_DIGEST_LENGTH and MD5_DIGEST_LENGTH in memcpy instead of hardcoded values. An incorrect value was used for MD5. - Fix message length bugs in libssh2_debug() There was a buffer overflow waiting to happen when a debug message was longer than 1536 bytes. Thanks to Daniel who spotted that there was a problem with the message length passed to a trace handler also after commit 0f0652a3093111fc7dac0205fdcf8d02bf16e89f. - Make libssh2_debug() create a correctly terminated string Also use FILE *stderr rather than fd 2, which can very well be something completely different. Daniel Stenberg (23 Jun 2010) - [TJ Saunders brought this change] handshake: Compression enabled at the wrong time In KEXINIT messages, the client and server agree on, among other things, whether to use compression. This method agreement occurs in src/kex.c's kex_agree_methods() function. However, if compression is enabled (either client->server, server->client, or both), then the compression layer is initialized in kex_agree_methods() -- before NEWKEYS has been received. Instead, the initialization of the compression layer should happen after NEWKEYS has been received. This looks to occur insrc/kex.c's diffie_hellman_sha1(), which even has the comment: /* The first key exchange has been performed, switch to active crypt/comp/mac mode */ There, after NEWKEYS is received, the cipher and mac algorithms are initialized, and that is where the compression should be initialized as well. The current implementation fails if server->client compression is enabled because most server implementations follow OpenSSH's lead, where compression is initialized after NEWKEYS. Since the server initializes compression after NEWKEYS, but libssh2 initializes compression after KEXINIT (i.e. before NEWKEYS), they are out of sync. Reported in bug report #180 - [TJ Saunders brought this change] userauth_hostbased_fromfile: packet length too short The packet length calculated in src/userauth.c's userauth_hostbased_fromfile() function is too short by 4 bytes; it forgets to add four bytes for the length of the hostname. This causes hostbased authentication to fail, since the server will read junk data. verified against proftpd's mod_sftp module - _libssh2_userauth_publickey: reject method names longer than the data This functions get the method length by looking at the first 32 bit of data, and I now made it not accept method lengths that are longer than the whole data set is, as given in the dedicated function argument. This was detected when the function was given bogus public key data as an ascii string, which caused the first 32bits to create a HUGE number. - NULL resistance: make more public functions survive NULL pointer input Sending in NULL as the primary pointer is now dealt with by more public functions. I also narrowed the userauth.c code somewhat to stay within 80 columns better. - agent: make libssh2_agent_userauth() work blocking properly previously it would always work in a non-blocking manner Peter Stuge (17 Jun 2010) - Fix underscore typo for 64-bit printf format specifiers on Windows Commit 49ddf447ff4bd80285f926eac0115f4e595f9425 was missing underscores. Daniel Stenberg (16 Jun 2010) - libssh2_session_callback_set: extended the man page