Troubleshooting SSL Protocols and Cipher Suites

Protocols and Ciphers

TLS and SSL contain several different protocol versions and utilize many different encryption cipher suites. In any particular connection between a client and a server, the two endpoints will agree on both a protocol and cipher to use for that particular connection. As such, every client and server is configured to support a particular set of the available protocols and ciphers. In cases where there are no available protocols/ciphers that both the client and server support, an SSL handshake error will occur and the connection will not be established. In AirWatch, the application logs will throw standard SSL/TLS errors if a handshake failure occurs.

Generally speaking, recent ciphers are more secure than older ciphers, and ciphers using a larger encryption key length are more secure than those with a smaller length. Additionally, as security holes are discovered in various protocols and ciphers, they may be deprecated, while new standards may be adopted over time as they are developed. Thus, as server and software updates are applied to various network components, it is possible that compatibility issues may arise that need to be resolved manually. The steps below will give you the ability to identify when these issues occur and to resolve them.


Identifying a Handshake Error with WireShark

The best way to occur an SSL/TLS handshake error has occurred is to use a network packet trace while the issue is occurring. The open-source application Wireshark (available here) is very useful in performing these tests. When Wireshark is run on a particular server, it will analyze all traffic passing through that server. Thus, by running Wireshark on the client server (for example, on a MAG or a SEG when analyzing a connection to AirWatch servers), you can get insight into the actual data exchange and any errors occurring. A packet trace from Wireshark will analyze all traffic between when the a trace is started and when it is stopped. After that, additional filters can be applied to narrow down the traffic that needs to be investigated. The basic practice for tracing useful information is:

  1. Prepare the test as much as possible. For example, if a test connection button is being tested, navigate to the proper webpage so that you can test the connection as soon as the trace is started. If a connection is failing as soon as a particular service is started, be ready to start that service as soon as the trace begins.
  2. Start the trace.
  3. Perform the action that is failing.
  4. Stop the trace. Ideally the time between starting and stopping the trace should be as short as possible to minimize irrelevant traffic.
  5. Apply filters and analyze the trace.

With this test, we are looking to identify if a handshake error has occurred, as well as to identify the cipher suites supported by the client. In order to filter the available traffic, we can apply a filter such as "ip.dst ==". This will only show packets initiated from the client server being sent to IP address Additionally, we can use the filter "ssl" to show only traffic using an SSL/TLS protocol. Together, the filter "ip.dst == && ssl" will apply both of these rules. Another option for filtering traffic to a specific stream is to find a single relevant packet, right-click it, and select follow TCP stream. This will automatically apply a filter that will show all traffic related to a single individual request between a client and server, and can be useful in identifying the specific responses to individual packets. A good strategy to find a relevant interaction is to first apply the previously mentioned filters, find a Client Hello packet, and then right-click that packet to follow the TCP stream.

In cases of a handshake failure, you will see a packet similar to that shown below. In this case the TCP stream has been filtered so the first packet is a client hello, followed by a response from the server indicating a TLS handshake failure. In this event, we will need to identify the supported protocols and cipher suites of both the client and server to identify the root cause of the handshake failure.


An example of another trace is shown below. Note that in this example, a handshake failure does not occur. When identifying encryption ciphers supported by the client, the best place is to look for the 'Client Hello' packet. Select this packet, and then expand Secure Sockets Layer > Handshake Protocol: Client Hello > Cipher Suites. The list shown here will be all available ciphers that the client supports. Additionally, a few lines above the cipher suites you will be able to see the SSL/TLS version used to initiate the handshake. During the handshake process the two servers will ultimately try to choose one of these ciphers, so as long as there is at least a single match the handshake should be successful.



Identifying the Available Cipher Suites for a Server

Now that we've identified the available ciphers on the client, we'll need to confirm the available ciphers on the server. During a successful handshake, the server will respond with a 'Server Hello' packet. This can be analyzed in the same way to determine the encryption cipher chosen to be used. However, during the event of a handshake failure, WireShark won't be sufficient in determining the available ciphers from the server. This info can be found directly from the server itself, or through various web tools for convenience or if you don't have physical access to the server. In particular, websites similar to Qualys' SSL Labs (found here) are useful in troubleshooting SSL/TLS issues.

After navigating to the website, enter in the public URL of the server you are trying to connect to. Several tests will then be performed and displayed on the web page. Under the 'Configuration' section, shown below, you will see a list of available SSL/TLS protocols and cipher suites. Verify that there is at least one protocol and cipher suite that is also supported by the client server. Generally, these will be listed in most-preferred order, so it is recommended to chose a more secure protocol to be used if possible.



Updating Available Protocols and Cipher Suites

Ultimately, if there is no supported cipher for both the client and server, at least one must be manually enabled on the client or server. This can be manually done through the registry, but a quicker and easier solution is to use a tool by Nartac Software called IIS Crypto, available here.

When opened, IIS Crypto will allow you to manage all available protocols and ciphers available for use on that server. Simply choose the appropriate options, and select 'Apply.' Note that before the changes will be applied, the server will need to be restarted. However, at that point the available protocols/ciphers will be updated and, if chosen correctly, any handshake errors will be resolved. If an error still occurs, the previous steps should be repeated to identify any changes in the behavior.



Notes about SSL/TLS protocols and ciphers

Many modern servers will support traffic using SSL 3.0, TLS 1.0, TLS 1.1, and TLS 1.2. protocols. However, earlier protocols, namely SSL 3.0, TLS 1.0, and TLS 1.1, may contain security vulnerabilities if enabled. Restricting a server to only utilize the latest protocols and cipher suites is often the most secure practice. However, when interfacing with other servers and enterprise applications, it is often necessary to support earlier protocols and cipher suites to ensure compatibility between applications. Generally during the handshake process the client and server will negotiate the most secure protocol available to both, even when less secure options are available.

Similarly, out of all available cipher suites, some contain known vulnerabilities, and some are just cryptographically weaker than other ciphers. When choosing a cipher suite to use, there are differences between both the encryption method itself as well as the key exchange mechanism. Additionally, there will also be differences in the key length used (ie: 256 bits vs 128 bits). An example cipher suite is below


Some notes about this cipher suite:

  • uses ECDHE RSA key exchange
  • uses AES encryption
  • uses 256 bit key length
  • uses the CBC mode of operation
  • uses a SHA384 hash function

Different cipher suites will use a different combination of each of these features, and all must be considered when determining the security of a cipher suite. However, generally AES encryption is considered a top standard, and larger keys will be more secure (with a trade off to performance). RC4 and 3DES are other commonly used encryption methods, but AES is preferred due to increased security and efficiency, respectively.

Other Languages: 日本語

Have more questions? Submit a request


Article is closed for comments.