Botan4 Tracking Issue

It took almost 2.5 years from when development stopped on Botan2 until Botan 3.0 was released. That was suboptimal.

Goal this time around is that the entire Botan3->Botan4 development phase happens in a single release cycle. In one quarter there is a Botan 3.y.0 feature release, same as every quarter. The next quarter sees the release of Botan 4.0.0, plus a Botan3 patch release as required.

This requires effectively pre-loading almost all of the development work in advance, such that once master moves to 4.0-pre the process is mostly just hitting merge-merge-merge on a pile of already existing PRs. Thus the very long lead time (and the desire for a memorable ticket number, since this meta issue will be open for years - still sad I missed out on #4444).

Note

For issues or questions related to specific subtasks, please open up a new issue for discussion, and reference this ticket. Otherwise, this issue is likely to become clogged with many unrelated conversations.

General Outline

Botan4 is still C++20. The gains we'd get from C++23/C++26 seem minimal (not zero, but nothing amazing) and language bumps do have an effect on end users. Hopefully MISRA eventually allows C++20...

Increasing compiler versions TBD. Certainly Clang minimum version increases, so we can use std::source_location (prior work done by @KaganCanSit that could be resurrected: #5084). Increasing minimum GCC to 14 would ensure we always have __builtin_addc which would be useful for optimization purposes. At least Clang 17 for the CWG 2518 fix.

Timeline

TBD. Maybe mid 2027?

Significant Development Work

The bigger projects

• Split public key and private key types
• Remove DHE support from TLS
• Remove RSA key exchange from TLS
• Remove CBC ciphers from TLS
• Post removal of kyber_90s and dilithium_aes some cleanup are possible
• Possibly ML-KEM vs Kyber and ML-DSA vs Dilithium cleanups
• Support large element OIDs
• Internal EC data cleanups after removing BigInt EC point logic
• BigInt public API cleanups
• Resurrect changes in Replace BOTAN_FFI_VISIT macro with std::source_location (use botan_ffi_visit function) #5084

For some of these it may be possible to do significant amounts of work on master beforehand to get ready, which will make life simpler. In particular for DHE/RSA/CBC in TLS it may be possible to just make them optional (ie tls does not hard depend on dh module); this improves things on master immediately for those who would prefer a smaller attack surface, and makes the final incompatible change easier. Splitting the key types and the ML-KEM/ML-DSA cleanups likewise I think a lot of prep can be done without violating SemVer.

Incompatible Changes

Misc mostly easy changes, no need to preload these can probably all be done over a weekend.

• Removing ~all deprecated functions/classes
• Remove the deprecated elliptic curve groups
• Remove the deprecated DL groups
• Decide how much of explicit curve support to drop #4684
• Remove serialization/deserialization of EC identity element
• Remove PBKDF and subclasses
• Hide PasswordHash headers
• Remove deprecated PK padding aliases
• Headers going internal: numthry.h, reducer.h, compiler.h
• Underscore prefix all internal functions (eg create_encryption_op) [also consider using some nasty _botan_internal_ prefix instead of just _]

Module Removal

If you, dear reader, are relying on any of these modules/algorithms in your code, please asap open a sub-issue for discussion, so we are aware of your usage. Do not reply to this ticket about specific modules.

• Decide if cryptobox should be removed in Botan4
• Decide if dilithium_aes should be removed in Botan4
• Decide if dlies should be removed in Botan4
• Decide if gost_28147 should be removed in Botan4
• Decide if gost_3411 should be removed in Botan4
• Should keccak support be removed in Botan4 #4721
• Decide if kyber_90s should be removed in Botan4
• Decide if legacy_ec_point should be removed in Botan4
• Decide if lion should be removed in Botan4
• Decide if mce should be removed in Botan4
• Decide if md4 should be removed in Botan4
• Decide if noekeon should be removed in Botan4
• Decide if shake_cipher should be removed in Botan4
• Decide if tpm (and also uuid) should be removed in Botan4
  • Note tpm is the TPMv1 only module, tpm2 supporting TPMv2 is staying

Currently deprecated but not removed

These are already deprecated and certainly not desirable but kept for Botan4

• crc32
• dsa
• gost_3410
• md5
• siphash
• streebog
• x919_mac

Proposed, now cancelled, incompatible changes

• Do not remove OtherNames support for certs #4678