通过http形式提供服务，有两大类，一是get形式，请求服务；二是post形式，push消息得到服务，以下使用不同语言实现get和post数据形式，基础源自于java和python，scala则是通过调用java的方式实现，hive中也是，其中hive需要自定义udf。
get服务
通过传入参数，发出请求，获取到数据，
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
| public String getData(String inputParams) {
String output=null;
String targetURL = "*****";
try {
targetURL = targetURL +"?" +inputParams;
URL restServiceURL = new URL(targetURL);
HttpURLConnection httpConnection = (HttpURLConnection) restServiceURL.openConnection();
httpConnection.setRequestMethod("GET");
httpConnection.setRequestProperty("Accept", "application/json");
httpConnection.setDoOutput(false);// 是否输入参数，参数已经当成url连接一部分了
if (httpConnection.getResponseCode() != 200) {
throw new RuntimeException("HTTP GET Request Failed with Error code : "
+ httpConnection.getResponseCode());
}
BufferedReader responseBuffer = new BufferedReader(new InputStreamReader(
(httpConnection.getInputStream())));
output = responseBuffer.readLine();
httpConnection.disconnect();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return output;
}
|
缺点是每次请求需要建立一次连接，并且需要把参数进行拼接好，a=1&b=2&c=3形式；
post数据
提交数据量大，不适于在url中当做参数传值；或者请求第三方服务，使用post方式提交数据获取服务的结果。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
| public String jsonPost(String strURL, String params) {
try {
URL url = new URL(strURL);// 创建连接
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setDoInput(true);
connection.setUseCaches(false);
connection.setInstanceFollowRedirects(true);
connection.setRequestMethod("POST"); // 设置请求方式
connection.setRequestProperty("Accept", "application/json"); // 设置接收数据的格式
connection.setRequestProperty("Content-Type", "application/json"); // 设置发送数据的格式
connection.connect();
OutputStreamWriter out = new OutputStreamWriter(connection.getOutputStream(), "UTF-8");
out.append(params);
out.flush();
out.close();
int code = connection.getResponseCode();
InputStream is = null;
if (code == 200) {
is = connection.getInputStream();
} else {
is = connection.getErrorStream();
}
// 读取响应
int length = (int) connection.getContentLength();// 获取长度
if (length != -1) {
byte[] data = new byte[length];
byte[] temp = new byte[512];
int readLen = 0;
int destPos = 0;
while ((readLen = is.read(temp)) > 0) {
System.arraycopy(temp, 0, data, destPos, readLen);
destPos += readLen;
}
String result = new String(data, "UTF-8");
return result;
}
} catch (IOException e) {
return "error";
}
return "error";
}
|
python的post方式
python的提交方式与java的原理一样，只是需要对中文进行编码，
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
|
import urllib
from urllib import parse
from urllib import request
data={}
data['cityId']=49
data['mid']=1
data['type']=2
url='******'
post_data = parse.urlencode(data).encode(encoding='UTF8')
print(post_data.decode('utf-8'))
req = request.urlopen(url, post_data)
content = req.read()
print(content.decode('utf-8'))
|
scala中post
在scala里调用java的方法即可，如果是批量，则最好先建立一个长连接。
hive的udf实现
hive的udf实现，实际上是java实现的，相当于在udf里新增一个post数据的方法，参考hive的udf写法。
