Expert Consultancy from Yellow Pelican

tHttpRequest

A site about Talend

tHttpRequest Component

The tHttpRequest component is part of the Internet family of components, and makes both POST and GET requests to the specified URI.

Image 1

Schema

tHttpRequest has a fixed Schema consisting of a single String field named RequestBody. For successful requests, RequestBody will contain the result of the request.

URI

URI allows you to specify the address of the resource required.

Method

tHttpRequest allows you to make both POST and GET requests. For POST request, you must specify a file that contains the Post parameters.

Write response content to file

As well as writing the request response as row data, tHttpRequest may also write the response to a file. Selecting this option allows you to specify the name of a file that the response should be written to.

Headers

Headers allows you to specify HTTP Header Fields, as value-pairs. This allows you to add or override the headers that are sent with the request.

Need authentication

If the service that is being requested, requires user and password authentication, select this option. This allows you to specify the user and password that will be used for the request.

Example #1

In this example, a GET request is used, to get a stock quotation from Yahoo! Finance.

As can be seen from the screenshot, below, a correctly formatted URI "http://finance.yahoo.com/d/quotes.csv?s=GOOG&f=snab" has been entered as a GET request.

Image 2

Example #1 Result

Starting job tHttpRequestExample at 07:41 17/10/2013.


[statistics] connecting to socket on port 4073
[statistics] connected
.------------------------------------.
|             tLogRow_1              |
|=----------------------------------=|
|ResponseContent                     |
|=----------------------------------=|
|"GOOG","Google Inc.",897.95,894.22
|
'------------------------------------'

[statistics] disconnected
Job tHttpRequestExample ended at 07:41 17/10/2013. [exit code=0]

Error Handling

Use of tHttpRequest does pose some Error Handling issues. We'll explore these in this article.

Exception Handling

Certain conditions will cause tHttpRequest to throw an Exception. If we re-run our example Job while we have no Internet connection, we'll see the following results.

Starting job tHttpRequestExample at 17:42 17/10/2013.


[statistics] connecting to socket on port 3785
[statistics] connected
Exception in component tHttpRequest_1
java.net.UnknownHostException: finance.yahoo.com
	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:223)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:431)
	at java.net.Socket.connect(Socket.java:527)
	at java.net.Socket.connect(Socket.java:476)
	at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:388)
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:523)
	at sun.net.www.http.HttpClient.(HttpClient.java:227)
	at sun.net.www.http.HttpClient.New(HttpClient.java:300)
	at sun.net.www.http.HttpClient.New(HttpClient.java:317)
	at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:970)
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:911)
	at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:836)
	at talendbyexample.thttprequestexample_0_1.tHttpRequestExample.tHttpRequest_1Process(tHttpRequestExample.java:566)
[statistics] disconnected
	at talendbyexample.thttprequestexample_0_1.tHttpRequestExample.runJobInTOS(tHttpRequestExample.java:875)
	at talendbyexample.thttprequestexample_0_1.tHttpRequestExample.main(tHttpRequestExample.java:740)
Job tHttpRequestExample ended at 17:42 17/10/2013. [exit code=1]

In this example, our Job has immediately terminated with java.net.UnknownHostException: finance.yahoo.com. Some components allow you to continue after an exception; tHttpRequest does not. We may choose to add our own Exception Handling. For more information on this, read our article on the tJavaFlex Component.

On Component Error

tHttpRequest supports the trigger On Component Error and this can be used in conjunction with the value Error Message - ERROR_MESSAGE (After). This value can be retieved from tGlobalMap (String) globalMap.get("tHttpRequest_1_ERROR_MESSAGE") when a Component Error occurs. Unfortunately, I have been unable to identify any event that causes this trigger to be fired.

HTTP Status Codes

On completion of a request being made, an HTTP Status Code will be returned, together with the requested content (assuming the request was successful).

This is where the component tHttpRequest lets us down, somewhat. To undersand why this is, let's look at some of the code that Talend will generate for our code.

...
if (java.net.HttpURLConnection.HTTP_OK == (urlConn_tHttpRequest_1
	.getResponseCode())) {
	java.io.InputStream bis_tHttpRequest_1 = new java.io.BufferedInputStream(
	urlConn_tHttpRequest_1.getInputStream());
	while ((bos_buffer_tHttpRequest_1 = bis_tHttpRequest_1
	.read(buffer_tHttpRequest_1)) != -1) {
	sb_tHttpRequest_1.append(new String(
	buffer_tHttpRequest_1, 0,
	bos_buffer_tHttpRequest_1));
	}
	bis_tHttpRequest_1.close();
} else {
	System.err
	.println(urlConn_tHttpRequest_1.getResponseCode()
		+ " "
		+ urlConn_tHttpRequest_1
		.getResponseMessage());
}
row1.ResponseContent = sb_tHttpRequest_1.toString();
...

As can be seen from the above code fragment, if an HTTP Status Code of HTTP_OK is returned, our requested content is written to sb_tHttpRequest_1 and, later, made available to our Job as row1.ResponseContent.

If any other HTTP Status Code of HTTP_OK is returned, the value urlConn_tHttpRequest_1.getResponseCode() + " " + urlConn_tHttpRequest_1.getResponseMessage() is written to the console System.err.println. Any information relating to the request error is lost to the Job Developer. All is known is that row1.ResponseContent is a null reference and we can deduce that the request has not been successful. Unfortunately (programmatically), we do not know why.

Next

For fur information on on how Exceptions may be handled, read our article on the tJavaFlex component. To see an example Job using this component (including an Exception handler), read our article Stock Market Analysis Project - Get Stock Quotes.




Expert Consultancy from Yellow Pelican
comments powered by Disqus

© www.TalendByExample.com