Certificates, more accurately keys, play a vital role in making an AS2 connection secure and trusted:
Sender signs the composed message using her secret private key. The receiver can use the publicly-known public key of the sender to verify this signature, to ensure that it actually came from the sender (authenticity).
Sender encrypts the signed message using the receiver’s publicly-known public key. Only the receiver can decrypt this content, using her corresponding secret private key (confidentiality).
Receiver may similarly sign the returned MDN receipt using her private key. Again, the sender can use the publicly-known public key of the receiver to verify this signature, to ensure that the MDN actually came from the receiver (authenticity); and also indirectly confirm that the original message was actually received (non-repudiation) by the receiver herself.
For above flow to work:
Both sender and receiver need their private keys, hence key pairs. (AS2, based on S/MIME, uses asymmetric cryptography so each private key is coupled to a public key.
Sender needs receiver’s public key, and vice versa.
In some cases, sender may use one key (pair) to sign messages, and a different key (pair) to decrypt incoming messages; if so, the receiver needs to know the public keys for both.
As described above, each party needs to know the public key(s) of the other. However a key by itself does not ensure complete trust; a malicious party may send you a public key on behalf of your actual counterparty, and would then be able to decrypt all traffic that you encrypt using that key.
As such, there needs to be a mechanism to verify the ownership of a public key - ideally in a global scope. This is the main purpose of a certificate.
A certificate contains the public key of some party (usually identified by a distinguished name (DN)), along with a signature made using the private key of a "trusted" certificate authority (CA). If you have the public key of the CA itself (again from some trusted source), you can verify the signature on the certificate. If verified, the certificate is proof that the CA has certified that the included public key belongs to the indicated party.
The CA public keys themselves are also distributed in this manner. There are a few globally trusted root CAs, whose certificates are signed by themselves (self-signed). Because they are globally trusted, many systems (e.g. web browsers, SSL/TLS tools, language runtimes) have built-in copies of these certificates (as the starting points or roots of trust). These CAs sign certificates of other CAs, who in turn sign those of others, and so forth - creating a trust chain.
A party (subject) can forward their public key to one of these CAs (issuer) - root or (usually) intermediate - and get it signed as a certificate. After that, any system that trusts the CA - or one of its "parents" (issuers) - would trust this new certificate. Such a trust anchor is automatically established as long as the system can find a valid trust chain upwards from the leaf (party/subject) to an issuer (CA) that it already trusts.
Similar to root CAs, a party can sign their public key using their own private key. Such a self-signed certificate would have the same identity (DN) as the subject (owner) and issuer (signer). In order to trust it, a system would need to explicitly import it to its trust store; however it does not rely on any other CA certificates, and so will be trusted right away - with a single-level trust anchor (root and leaf being the same).
As long as you have received a self-signed certificate via a trusted communication channel, it is safe for use in AS2 i.e. S/MIME. Because partners need to exchange their connection parameters anyway during the initial set-up, they generate and share self-signed certificates as a common practice.
You can generate a keypair with a self-signed certificate for use with your AS2 Gateway trading stations, using the Generate Certificate option on the Certificate Manager page.
Some partners may reject self-signed certificates and demand you to provide a CA-issued certificate (with other specified parameters such as signature algorithm and min/max validity period). In such cases, you need to:
contact a CA and send them a certificate signing request (CSR) for your public key (along with any other parameters),
share the returned certificate with your trading partner, and
assign it to the AS2 configurations - in sync with your partner (both parties should switch to using the new keypair, at the same point in time).
Generating the CSR and updating the received certificate, can be done from AS2 Gateway’s Certificate Manager page itself, using the Renew Certificate option.
A key store is a container (usually a single file, similar to a zip archive) that can hold multiple private keys and/or certificates. They have two common use cases:
Holding a private key (identity) with the corresponding public certificate/chain; an identity store
Holding a set of (trusted) certificates, such as those of CAs; a trust store
In a key store, each entity is stored under an alias (alphanumeric identifier). In case of key pairs, the private key and the public certificate (or chain) are stored under the same alias; the key store reader utility will return the correct entry when requested by the corresponding type.
Key stores come in a few main types:
.jks - Java Key Store, Java’s proprietary format; widely being replaced by P12
.p12 - PKCS#12, the present industry standard
.pfx - Personal Information Exchange, predecessor of PKCS#12 (usually contains data in PKCS#12 format); common in Windows platform
Key stores are usually protected by a password; it is mainly for integrity (verifying the key store has not been tampered with) rather than confidentiality (i.e. not for encrypting the content). So, even if you do not have the password, you can still list/read/export the public content (public keys, certificates) inside a key store. However, private (identity) keys inside a key store are protected again with a password (which usually differs from the keystore password), and you cannot extract them without having the password.