Chapter 3 HTTP

網路連結下，電腦與電腦如何講悄悄話來傳遞訊息？

3.1 學習目標

• 了解網路資料傳輸協定HTTP。

• 在R裡如何進行HTTP資料傳輸。

• API應用實作：LINE，Facebook等。

Read:

• httr: https://httr.r-lib.org/

Reference:

• Google Diagflow: https://dialogflow.com/

  • languages support: https://cloud.google.com/dialogflow/docs/reference/language

3.2 網路通訊協定 HTTPS (HTTP)

an extension of the Hypertext Transfer Protocol (HTTP). It is used for secure communication over a computer network, and is widely used on the Internet.

• HTTP:

  an application protocol for distributed, collaborative, hypermedia information systems. HTTP is the foundation of data communication for the World Wide Web, where hypertext documents include hyperlinks to other resources that the user can easily access, for example by a mouse click or by tapping the screen. HTTP was developed to facilitate hypertext and the World Wide Web.

3.2.1 Request message

A typical request message that follows HTTP should have:

• a request line (e.g., GET /images/logo.png HTTP/1.1, which requests a resource called /images/logo.png from the server.)

  範例：

  資源路徑：/orgs/octokit/repos
  資源伺服器網址： https://api.github.com
  點https://api.github.com/orgs/octokit/repos可取得所要資源

資源伺服器網址也叫做root endpoint。

• request header fields (e.g., Accept-Language: en).

• an empty line

• an optional message body

3.2.2 Request methods

Current HTTP allows GET, PUT, POST, DELETE methods, etc. Among them,

• GET: 取得資源（資料）

• POST: 新增資源

3.2.3 Response message

• a status line which includes the status code and reason message (e.g., HTTP/1.1 200 OK, which indicates that the client’s request succeeded.)

• response header fields (e.g., Content-Type: text/html)

• an empty line

• an optional message body

我們平日所使用的瀏覽器是在進行GET請求，得到訊息後瀏覽器再以我們可理解文字圖像呈現在眼前。

library(httr)
GET("https://www.ntpu.edu.tw") -> res
head(res$content) # 網頁內容（已編碼，要進一步還原成可理解內容）

Response content常見有以下三種格式：

• 文字：網址回傳一個文字檔時。

• JSON：網址回傳一串文字資訊，如REST API。

• Binary：網址回傳一個「非文字檔」。

3.3 curl

curl (client URL): a command-line tool for getting or sending data including files using URL syntax.

• 讓個人電腦也可以進行各網路通訊協定的資訊交換。

• 使用terminal視窗

curl https://www.ntpu.edu.tw

3.4 httr

R裡用來操作curl的套件。

3.5 httr: curl online translator

https://curl.trillworks.com/

• 可翻譯多語言

3.6 httr: curlconverter

R套件，也是RStudio Addins，只要電腦裡有複製的cur語句（不一定要在RStudio裡）, 它會幫你翻成httr語法。

3.6.1 clipr::read_clip()

library(psych)
read.clipboard() 

library(clipr)
read_clip() -> curlOrigin

curlOrigin %>%
  str_replace("(\\{channel access token\\})","token")

3.6.2 不使用addins

安裝

devtools::install_github("hrbrmstr/curlconverter")

啟用

library(curlconverter)
library(httr)

httpbinrhcurl <- "curl 'https://httpbin.org/headers' -H 'pragma: no-cache' -H 'accept-encoding: gzip, deflate, sdch' -H 'accept-language: en-US,en;q=0.8' -H 'upgrade-insecure-requests: 1' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.39 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'cache-control: no-cache' -H 'referer: https://httpbin.org/' --compressed"

取出原始curl語法所需訊息：

straight <- straighten(httpbinrhcurl)

翻譯成httr的對應request message:

• make_req: turn parsed cURL command lines into a httr request functions

res <- make_req(straight) # 回傳res是個list of function

查看對應httr語法要看res裡的function內容

res[[1]]

body(res[[1]])

進行httr request

res[[1]]() -> responseMessage

Request有時可能會失敗，但成功的話responseMessage$status_code會是200.

• 詳細HTTP status code

3.6.3 完整流程

library(dplyr); library(magrittr); library(curlconverter)
httpbinrhcurl %>% # 原始curl語法
  stringr::str_c(collapse="") %>% # 把curl command合成一行
  stringr::str_replace_all("\\\\","") %>% # 去除換行符號 \
  straighten() %>% # 取出httr所需資訊
  make_req() %T>% # 翻譯成httr對應的HTTP verb呼叫方式
    {assign("requestMessage", ., envir = globalenv())}  %>% # 存在requestMessage 
  {.[[1]]()} -> # 執行request
  responseMessage # 儲存response message

3.7 LINE Messaging API

Reference: https://developers.line.biz/en/reference/messaging-api/#send-reply-message

E-Major聊天機器人

3.7.1 Push messages to user

3.7.1.1 Using UserId

curl -v -X POST https://api.line.me/v2/bot/message/push \
-H 'Content-Type:application/json' \
-H 'Authorization: Bearer cn2ntQne/ZGklb3rJkDBi6zIgSpgjQsHpyZ+MiwtV1l8/13kV6INHyXFyMN1Ygk91HrtZ4NQ6kjq9AqfOjshmEGaI4ipNRYIRjVd1z1EFZD7y1d5cGUOlcGfqxl5l3TIL5fNXRVUPN+54K3+r63eCgdB04t89/1O/w1cDnyilFU=' \
-d '{
    "to": "Uee05ee6b5f5140979d9c62017e2297b3",
    "messages":[
        {
            "type":"text",
            "text":"Hello, world1"
        },
        {
            "type":"text",
            "text":"Hello, world2"
        }
    ]
}'

3.7.1.2 使用online translator

將前一節內容貼到https://curl.trillworks.com/可得到以下程式碼：

headers = c(
  `Content-Type` = 'application/json',
  `Authorization` = 'Bearer cn2ntQne/ZGklb3rJkDBi6zIgSpgjQsHpyZ+MiwtV1l8/13kV6INHyXFyMN1Ygk91HrtZ4NQ6kjq9AqfOjshmEGaI4ipNRYIRjVd1z1EFZD7y1d5cGUOlcGfqxl5l3TIL5fNXRVUPN+54K3+r63eCgdB04t89/1O/w1cDnyilFU='
)

data = '{\n    "to": "Uee05ee6b5f5140979d9c62017e2297b3",\n    "messages":[\n        {\n            "type":"text",\n            "text":"Hello, world1"\n        },\n        {\n            "type":"text",\n            "text":"Hello, world2"\n        }\n    ]\n}'

res <- httr::POST(url = 'https://api.line.me/v2/bot/message/push', httr::add_headers(.headers=headers), body = data)

3.7.1.3 使用curlconverter (有誤)

3.8 HERE geo API

• https://developer.here.com/

3.9 Goodread API

• https://www.goodreads.com/api

3.10 Facebook API

• demo chatroom: https://www.messenger.com/t/OriginalCoastClothing

3.11 GitHub API

• https://developer.github.com/v3/

3.12 大數據競賽

• 國道高速公路：https://www.freewaytw.com/data?fbclid=IwAR2bL4mxEeAc1sDH9q7xEF1tUr_WvMIZh-6DhfATD7JHaukuMWlV-1ihPSc

3.13 Hypothes.is project

• 將自己的private highlights打開成public

• 抓取學生自己的highlights

• 整合成全班的highlights

• API reference: https://h.readthedocs.io/en/latest/api/
  https://h.readthedocs.io/en/latest/api-reference/v1/