HTTP Client can provide synchronous and asynchronous request mechanisms via the following three core classes:
- HttpRequest: Request that needs to be sent via the HttpClient.
- HttpClient: It is a container for multiple requests.
- HttpResponse: All requests need to complete the cycle and provide the result as HttpResponse. Each response is identified with the status codes like
200 -> Response obtained and it is OK means everything is correct 404 -> Requested page or element not available in the mentioned URL 401 -> Unauthorized
In this tutorial let us see some of the ways to test a httpClient that contains headers and SSL.
Example Project
Project Structure:
As this is a maven project, dependencies are added in pom.xml
XML
<?xml version="1.0" encoding="UTF-8"?> xsi:schemaLocation="https://maven.apache.org/POM/4.0.0 <modelVersion>4.0.0</modelVersion> <artifactId>httpclient-sampletesting</artifactId> <version>0.1-SNAPSHOT</version> <name>httpclient-sampletesting</name> <packaging>war</packaging> <parent> <groupId>com.gfg</groupId> <artifactId>parent-spring-5</artifactId> <version>0.0.1-SNAPSHOT</version> <relativePath>../parent-spring-5</relativePath> </parent> <dependencies> <dependency> <groupId>org.apache.httpcomponents</groupId> <artifactId>httpcore</artifactId> <version>${httpcore.version}</version> <exclusions> <exclusion> <artifactId>commons-logging</artifactId> <groupId>commons-logging</groupId> </exclusion> </exclusions> </dependency> <!-- utils --> <dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-lang3</artifactId> <version>${commons-lang3.version}</version> </dependency> <!-- http client --> <dependency> <groupId>org.apache.httpcomponents</groupId> <artifactId>httpclient</artifactId> <version>${httpclient.version}</version> <exclusions> <exclusion> <artifactId>commons-logging</artifactId> <groupId>commons-logging</groupId> </exclusion> </exclusions> </dependency> <dependency> <groupId>org.apache.httpcomponents</groupId> <artifactId>fluent-hc</artifactId> <version>${httpclient.version}</version> <exclusions> <exclusion> <artifactId>commons-logging</artifactId> <groupId>commons-logging</groupId> </exclusion> </exclusions> </dependency> <dependency> <groupId>org.apache.httpcomponents</groupId> <artifactId>httpmime</artifactId> <version>${httpclient.version}</version> </dependency> <dependency> <groupId>commons-codec</groupId> <artifactId>commons-codec</artifactId> <version>${commons-codec.version}</version> </dependency> <dependency> <groupId>org.apache.httpcomponents</groupId> <artifactId>httpasyncclient</artifactId> <version>${httpasyncclient.version}</version> <exclusions> <exclusion> <artifactId>commons-logging</artifactId> <groupId>commons-logging</groupId> </exclusion> </exclusions> </dependency> <dependency> <groupId>com.github.tomakehurst</groupId> <artifactId>wiremock</artifactId> <version>${wiremock.version}</version> <scope>test</scope> </dependency> <!-- web --> <dependency> <groupId>javax.servlet</groupId> <artifactId>javax.servlet-api</artifactId> <version>${javax.servlet-api.version}</version> <scope>provided</scope> </dependency> <dependency> <groupId>javax.servlet</groupId> <artifactId>jstl</artifactId> <version>${jstl.version}</version> <scope>runtime</scope> </dependency> </dependencies> <build> <finalName>httpclient-simple</finalName> <resources> <resource> <directory>src/main/resources</directory> <filtering>true</filtering> </resource> </resources> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-war-plugin</artifactId> <version>${maven-war-plugin.version}</version> </plugin> <!-- This should be added to overcome Could not initialize class org.apache.maven.plugin.war.util.WebappStructureSerializer --> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-war-plugin</artifactId> <version>3.3.2</version> </plugin> </plugins> </build> <profiles> <profile> <id>live</id> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <executions> <execution> <phase>integration-test</phase> <goals> <goal>test</goal> </goals> <configuration> <excludes> <exclude>none</exclude> </excludes> </configuration> </execution> </executions> </plugin> </plugins> </build> </profile> </profiles> <properties> <!-- util --> <commons-codec.version>1.10</commons-codec.version> <httpasyncclient.version>4.1.4</httpasyncclient.version> <!-- testing --> <wiremock.version>2.5.1</wiremock.version> <httpcore.version>4.4.11</httpcore.version> <httpclient.version>4.5.8</httpclient.version> <!-- maven plugins --> <cargo-maven2-plugin.version>1.6.1</cargo-maven2-plugin.version> </properties> </project> |
From HttpClient 4.3, requests can be built by using RequestBuilder. HTTP header can be added as mentioned in the below lines of code. Sample of creation of RequestBuilder and usage with content type
Java
CloseableHttpClient httpClient = HttpClients.custom().build();final HttpGet request = new HttpGet(GFG_URL);request.setHeader(HttpHeaders.CONTENT_TYPE, "application/json");httpResponse = httpClient.execute(request); |
Before HttpClient 4.3, we need to depend on the below kind of code. Any custom header can be set on a request by a simple setHeader call on the request:
Java
HttpClient httpClient = new DefaultHttpClient();HttpGet getRequest = new HttpGet(GFG_URL);request.setHeader(HttpHeaders.CONTENT_TYPE, "application/json");httpClient.execute(getRequest); |
Let us see the whole java test code for the versions from HttpClient version 4.3 onwards. Let us check out ResponseUtil.java as it is getting used
ResponseUtil.java
Java
import org.apache.http.HttpEntity;import org.apache.http.client.methods.CloseableHttpResponse;import java.io.IOException; public final class ResponseUtil { private ResponseUtil() { } public static void closeResponse(CloseableHttpResponse response) throws IOException { if (response == null) { return; } try { final HttpEntity entity = response.getEntity(); if (entity != null) { entity.getContent().close(); } } finally { response.close(); } }} |
SampleHttpClientHeadersUnitTest.java
Here we are passing the “Lazyroar” URL as request by using headers
Java
import com.google.common.collect.Lists;import org.apache.http.Header;import org.apache.http.HttpHeaders;import org.apache.http.client.ClientProtocolException;import org.apache.http.client.methods.CloseableHttpResponse;import org.apache.http.client.methods.HttpGet;import org.apache.http.client.methods.HttpUriRequest;import org.apache.http.client.methods.RequestBuilder;import org.apache.http.impl.client.CloseableHttpClient;import org.apache.http.impl.client.HttpClientBuilder;import org.apache.http.impl.client.HttpClients;import org.apache.http.message.BasicHeader;import org.junit.After;import org.junit.Before;import org.junit.Test; import static org.hamcrest.CoreMatchers.equalTo;import static org.junit.Assert.assertThat; import java.io.IOException;import java.util.List; public class SampleHttpClientHeadersUnitTest { private CloseableHttpClient httpClient; private CloseableHttpResponse httpResponse; @Before public final void before() { httpClient = HttpClientBuilder.create().build(); } @After public final void after() throws IllegalStateException, IOException { ResponseUtil.closeResponse(httpResponse); } // tests - headers - deprecated @Test public final void usageOfCustomUserAgentCheck() throws ClientProtocolException, IOException { httpClient = HttpClients.custom().setUserAgent("Mozilla/5.0 Firefox/26.0").build(); final HttpGet getRequest = new HttpGet(GFG_URL); httpResponse = httpClient.execute(getRequest); assertThat(httpResponse.getStatusLine() .getStatusCode(), equalTo(200)); } // tests - headers - user agent @Test public final void usageOfCustomUserAgentCheckWithConfig() throws ClientProtocolException, IOException { httpClient = HttpClients.custom().build(); final HttpGet request = new HttpGet(GFG_URL); request.setHeader(HttpHeaders.USER_AGENT, "Mozilla/5.0 Firefox/26.0"); httpResponse = httpClient.execute(request); assertThat(httpResponse.getStatusLine() .getStatusCode(), equalTo(200)); } @Test public final void usageOfCustomUserAgentCheckWithConfig1() throws ClientProtocolException, IOException { httpClient = HttpClients.custom().setUserAgent("Mozilla/5.0 Firefox/26.0").build(); httpResponse = httpClient.execute(new HttpGet(GFG_URL)); assertThat(httpResponse.getStatusLine() .getStatusCode(), equalTo(200)); } // tests - headers - content type @Test public final void usageOfCustomUserAgentCheckWithContentType() throws ClientProtocolException, IOException { httpClient = HttpClients.custom().build(); final HttpGet request = new HttpGet(GFG_URL); request.setHeader(HttpHeaders.CONTENT_TYPE, "application/json"); httpResponse = httpClient.execute(request); assertThat(httpResponse.getStatusLine() .getStatusCode(), equalTo(200)); } @Test public final void usageOfCustomUserAgentCheckWithContentType1() throws ClientProtocolException, IOException { final CloseableHttpClient client2 = HttpClients.custom().build(); final HttpGet request = new HttpGet(GFG_URL); request.setHeader(HttpHeaders.CONTENT_TYPE, "application/json"); httpResponse = client2.execute(request); assertThat(httpResponse.getStatusLine() .getStatusCode(), equalTo(200)); } @Test public final void usageOfCustomUserAgentCheckWithContentType2() throws ClientProtocolException, IOException { httpClient = HttpClients.custom().build(); final HttpUriRequest request = RequestBuilder.get().setUri(GFG_URL).setHeader(HttpHeaders.CONTENT_TYPE, "application/json").build(); httpResponse = httpClient.execute(request); assertThat(httpResponse.getStatusLine() .getStatusCode(), equalTo(200)); } @Test public final void usageOfCustomUserAgentCheckWithContentType3() throws ClientProtocolException, IOException { final Header header = new BasicHeader(HttpHeaders.CONTENT_TYPE, "application/json"); final List<Header> headers = Lists.newArrayList(header); httpClient = HttpClients.custom().setDefaultHeaders(headers).build(); final HttpUriRequest request = RequestBuilder.get().setUri(GFG_URL).build(); httpResponse = httpClient.execute(request); assertThat(httpResponse.getStatusLine() .getStatusCode(), equalTo(200)); } } |
Output on running as JUnit:
Next, let us see how to configure the Apache HttpClient 4 with SSL support. Instead of going as automated testing, let it be manually run as usual for any secured site, valid certificate is needed but in order to demonstrate how an SSL Live site can be tested via HttpClient, is shown without considering the certificate.
SampleHttpsClientSslLiveTest.java
Java
import static org.hamcrest.CoreMatchers.equalTo;import static org.junit.Assert.assertThat; import java.io.IOException;import java.security.GeneralSecurityException; import javax.net.ssl.SSLContext;import javax.net.ssl.SSLHandshakeException; import org.apache.http.HttpResponse;import org.apache.http.client.methods.HttpGet;import org.apache.http.config.Registry;import org.apache.http.config.RegistryBuilder;import org.apache.http.conn.socket.ConnectionSocketFactory;import org.apache.http.conn.ssl.NoopHostnameVerifier;import org.apache.http.conn.ssl.SSLConnectionSocketFactory;import org.apache.http.conn.ssl.TrustStrategy;import org.apache.http.impl.client.CloseableHttpClient;import org.apache.http.impl.client.HttpClientBuilder;import org.apache.http.impl.client.HttpClients;import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;import org.apache.http.ssl.SSLContextBuilder;import org.apache.http.ssl.SSLContexts;import org.junit.Test; public class SampleHttpsClientSslLiveTest { // tests @Test(expected = SSLHandshakeException.class) public final void checkForHttpsUrl() throws IOException { final CloseableHttpClient httpClient = HttpClientBuilder.create() .build(); final HttpGet getMethod = new HttpGet(SSL_HOST_URL); final HttpResponse response = httpClient.execute(getMethod); assertThat(response.getStatusLine() .getStatusCode(), equalTo(200)); } @Test public final void checkForHttpsUrl1WithCertificate() throws IOException, GeneralSecurityException { final TrustStrategy acceptingTrustStrategy = (certificate, authType) -> true; final SSLContext sslContext = SSLContexts.custom() .loadTrustMaterial(null, acceptingTrustStrategy) .build(); final SSLConnectionSocketFactory sslsocketFactory = new SSLConnectionSocketFactory(sslContext, NoopHostnameVerifier.INSTANCE); Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory> create().register("https", sslsocketFactory).build(); PoolingHttpClientConnectionManager clientConnectionManager = new PoolingHttpClientConnectionManager(socketFactoryRegistry); final CloseableHttpClient closeableHttpClient = HttpClients.custom() .setSSLSocketFactory(sslsocketFactory) .setConnectionManager(clientConnectionManager) .build(); final HttpGet httpGetMethod = new HttpGet(SSL_HOST_URL); final HttpResponse httpResponse = closeableHttpClient.execute(httpGetMethod); assertThat(httpResponse.getStatusLine() .getStatusCode(), equalTo(200)); closeableHttpClient.close(); } @Test public final void checkForHttpsUrlByIgnoringCertificate() throws Exception { final SSLContext sslContext = new SSLContextBuilder().loadTrustMaterial(null, (certificate, authType) -> true) .build(); final CloseableHttpClient client = HttpClients.custom() .setSSLContext(sslContext) .setSSLHostnameVerifier(new NoopHostnameVerifier()) .build(); final HttpGet httpGet = new HttpGet(SSL_HOST_URL); httpGet.setHeader("Accept", "application/xml"); final HttpResponse response = client.execute(httpGet); assertThat(response.getStatusLine() .getStatusCode(), equalTo(200)); } } |
On running the above code, we will see
Conclusion
Always it is good to write test cases for any scenario in the software lifecycle. We may need to interact with any website URL at any point in time or retrieve the information from it. In Java, using HttpClient can be handled wisely it can be handled by using the test cases above.
