Added MSRV

Added docs for oauth2 API
Added extra field from gitlab oauth2 response
Removed hw specific cargo build flag

.cargo/config.toml

-[build]
-rustflags = [
-  "-Ctarget-feature=+avx2", # recommended by crate chacha20poly1305
-  "-Ctarget-cpu=haswell", # recommended by crate chacha20poly1305
-]

README.md

@@ -17,7 +17,8 @@ See documentation on:
 * [Schema synchronization](./docs/Synchronization.md) between clients/plugins and the Pod
 * Performance [Measurement](./docs/PerformanceMeasurement.md)
 
-
+## MSRV (Minimal Supported Rust Version)
+Rust must be at least in version `1.60.0`. However we encourage to use the latest version available.
 ## Build & Run
 There are 3 main ways to run Pod: using pre-built docker images to just run it,
 building it in docker, and building it locally/natively.
@@ -44,26 +45,15 @@ This is the fastest way to compile Pod from source,
 for example, if you're making any changes in Pod and want to test it.
 It will also work on any OS and CPU architecture.
 
-You will need Rust >= 1.45 and sqlcipher:
-
-* On MacOS: `brew install rust sqlcipher`
-* On ArchLinux: `pacman -S --needed rust sqlcipher base-devel`
-* On Ubuntu and Debian:
-```
-apt-get install sqlcipher libsqlcipher-dev build-essential
-curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
-```
-
-After this, you can run Pod with:
 ```sh
 ./examples/run_development.sh
 ```
-### Make POD build faster
+### Make POD build faster (linux)
 It's possible to compile POD faster by switching the linker executable from default `GCC LD` to `Clang's LLD` or `mold`.
 1) Install alternative linker on your machine
 2) Create a `.cargo` directory in the `pod` or any in it's parent dirs (can be also in user home directory).
 3) Inside `.cargo` create a `config` file with following content:
-    ```
+    ```toml
     [build]
     incremental = true
     # use mold or lld linker, at your preference
@@ -78,13 +68,6 @@ To build only library and face fast compilation times, call
 
 
 ## Pod development
-If you develop Pod, you might want to have faster build turn-around.
-
-Use this to incrementally compile the project (after installing [cargo-watch](https://github.com/passcod/cargo-watch)):
-```sh
-cargo watch --ignore docs -s 'cargo check'
-```
-
 You can read about various components of the server:
 
 * Memri project: [blog.memri.io](https://blog.memri.io/)

docs/HTTP_API.md

@@ -14,9 +14,9 @@ This is typically the `owner_key` and `auth` keys in the request, see below.
 The `owner_key` tells the Pod *who* you claim to be.
 It is the full client's public [ED25519](https://ed25519.cr.yp.to/) key
 (see also [wikipedia](https://en.wikipedia.org/wiki/Curve25519)).
-It should be encoded as 64-character hex string.  
+It should be encoded as 64-character hex string.
 Pods configured with "ANY" as their owner will accept requests from any `owner_key`.
-For debugging, you can fake `owner_key` by providing any 64 hex characters, e.g. all zeroes.  
+For debugging, you can fake `owner_key` by providing any 64 hex characters, e.g. all zeroes.
 Note that for now, we don't really use any cryptography for the `owner_key`,
 but in the future we probably will.
 
@@ -119,7 +119,7 @@ The edge schema is defined through `ItemEdgeSchema` items. It is not possible to
 }
 ```
 
-The above json would define a directed edge `Message --sentBy--> Account`. Edges are defined separately for each `sourceType`, which means that a different `sentBy` schema could be defined for a `sourceType` that is not a `Message`. 
+The above json would define a directed edge `Message --sentBy--> Account`. Edges are defined separately for each `sourceType`, which means that a different `sentBy` schema could be defined for a `sourceType` that is not a `Message`.
 
 The Pod allows edges with multiple target types (type unions). These unions can be defined in the pod by simply creating multiple `ItemEdgeSchema` items with the same `edgeName` and `sourceType`. For example, an edge with target types `Message --sentBy--> [Account | Person]` should be saved in the pod by two `ItemEdgeSchemas`, where `targetType` is `Account` for the first item, and `Person` for the second item.
 
@@ -166,7 +166,7 @@ Create a single item.
 
 Returns an error if such `id` already exist in the DB.
 Returns an error if the new item doesn't conform to the Schema.
-Returns `id` of the created item if the operation is successful. 
+Returns `id` of the created item if the operation is successful.
 
 
 ### POST /v4/$owner_key/update_item
@@ -240,7 +240,7 @@ Returns an array empty array if either the element does not exist or if it has n
   }
 }
 ```
-Create a single directed edge item from an already existing item to another already existing item.  
+Create a single directed edge item from an already existing item to another already existing item.
 ⚠️ UNSTABLE: As of the current implementation, an additional `type="Edge"` item will be created
 holding the additional properties of the edge. This implementation detail is unstable,
 in the future we might either keep "Edge" as the item type name, or potentially introduce
@@ -372,7 +372,7 @@ Fetch custom data using GraphQL standards. Returns data in the query structure p
       // ...
     ],
     "deleteItems": [ "$id", "$id", "$id", /* ... */ ],
-    "createEdges": [ 
+    "createEdges": [
       {}, // same structure as create_edge endpoint above
       {},
       // ...
@@ -423,7 +423,7 @@ e.g. permission limitation.
 ```
 Get the logs of a pluginrun container, specified by the pluginrun `id`
 
-Returns 
+Returns
 ```json
 {
   "logs": "<log_string>"
@@ -498,3 +498,60 @@ Returns 404 NOT FOUND if the file has not been uploaded yet.
 
 Returns the contents of the file if the item describing the file exists and the file
 has already been uploaded to the Pod.
+
+# Oauth2 API
+### POST /v4/$OWNER_KEY/oauth2/auth_url
+```json
+{
+  "auth": $auth_json,
+  "payload": {
+      "scopes": list, platform specific,
+      "redirectUri": URL, string,
+      "platform": currently "gitlab" only supported
+  }
+}
+```
+
+Creates an authorization url that should be presented to the user.
+Example response:
+```
+https://gitlab.memri.io/oauth/authorize?response_type=code&client_id=fd94965d76207c614a2798feaf78666e38364078a256a8f31e20e4773fabb018&state=EfqI60oIcFxhi8IDaJnTHw&redirect_uri=http%3A%2F%2Flocalhost%3A8080&scope=read_user
+```
+
+### POST /v4/$OWNER_KEY/oauth2/access_token
+```json
+{
+  "auth": $auth_json,
+  "payload": {
+      "authCode": code from auth_url response,
+      "redirectUri": must be the same as for auth_url request,
+      "platform": "gitlab"
+    }
+}
+```
+
+Creates an access token for given authCode, example response
+```json
+{
+  "accessToken": "da9268a4d92c25a4e9c8e72b0952bea990b77775d6c785e4d660a89785d3ad7f",
+  "tokenType": "bearer",
+  "refreshToken": "42715aad0a47ad4b3850fc3c4f3b64b41e61af11ff8be2e57a1c70d12d0b3c63",
+  "expiresIn": 6749,
+  "createdAt": 1607635748
+}
+```
+
+### POST /v4/$OWNER_KEY/oauth2/refresh_token
+```json
+
+{
+  "auth": $auth_json,
+  "payload": {
+      "refreshToken": token from access_token response,
+      "redirectUri": must be the same as for auth_url request,
+      "platform": "gitlab"
+    }
+}
+```
+
+Creates new `access_token` from given `refresh_token`

examples/run_development.sh

@@ -8,7 +8,7 @@ if ! test -e Cargo.toml; then
 fi
 
 if ! test -v RUST_LOG; then
-  export RUST_LOG=pod=debug,info
+  export RUST_LOG=pod=debug,libpod=debug,info
 fi
 
 cargo build

libpod/src/api_model.rs

@@ -111,7 +111,7 @@ pub struct OauthAccessTokenPayload {
 }
 
 #[derive(Debug, PartialEq, Eq, Hash, Deserialize)]
-#[serde(rename_all = "lowercase")]
+#[serde(rename_all = "snake_case")]
 pub enum Platforms {
     Gitlab,
 }
@@ -143,6 +143,8 @@ pub struct Oauth2AccessTokenResponse {
     pub refresh_token: Option<String>,
     #[serde(skip_serializing_if = "Option::is_none")]
     pub expires_in: Option<u64>,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub created_at: Option<u64>,
 }
 
 #[derive(Deserialize, Debug)]

libpod/src/oauth2_api.rs

@@ -7,18 +7,44 @@ use crate::{
 };
 use lazy_static::lazy_static;
 use oauth2::{
-    basic::BasicClient, reqwest::async_http_client, AuthUrl, AuthorizationCode, ClientId,
-    ClientSecret, CsrfToken, RedirectUrl, RefreshToken, Scope, TokenResponse, TokenUrl,
+    basic::{
+        BasicErrorResponse, BasicRevocationErrorResponse, BasicTokenIntrospectionResponse,
+        BasicTokenType,
+    },
+    reqwest::async_http_client,
+    AuthUrl, AuthorizationCode, Client, ClientId, ClientSecret, CsrfToken, EmptyExtraTokenFields,
+    ExtraTokenFields, RedirectUrl, RefreshToken, Scope, StandardRevocableToken,
+    StandardTokenResponse, TokenResponse, TokenUrl,
 };
 use reqwest::StatusCode;
+use serde::{Deserialize, Serialize};
 use std::{collections::HashMap, env};
 use tracing::debug;
 
+type CustomClient<PlatformExtras> = Client<
+    BasicErrorResponse,
+    StandardTokenResponse<PlatformExtras, BasicTokenType>,
+    BasicTokenType,
+    BasicTokenIntrospectionResponse,
+    StandardRevocableToken,
+    BasicRevocationErrorResponse,
+>;
+
+type CustomTokenResponse<PlatformExtras> = StandardTokenResponse<PlatformExtras, BasicTokenType>;
+
+/// Non-standard oauth2 fields reported by Gitlab
+#[derive(Serialize, Debug, Deserialize)]
+struct GitlabExtras {
+    created_at: u64,
+}
+
+impl ExtraTokenFields for GitlabExtras {}
+
 pub async fn auth_url(payload: Oauth2AuthUrlRequest) -> Result<String> {
     debug!("Creating auth url for {:?}", payload.platform);
 
-    let client =
-        get_client(payload.platform)?.set_redirect_uri(RedirectUrl::new(payload.redirect_uri)?);
+    let client = get_client::<EmptyExtraTokenFields>(payload.platform)?
+        .set_redirect_uri(RedirectUrl::new(payload.redirect_uri)?);
 
     let (authorize_url, _csrf_state) = client
         .authorize_url(CsrfToken::new_random)
@@ -31,21 +57,19 @@ pub async fn auth_url(payload: Oauth2AuthUrlRequest) -> Result<String> {
 pub async fn access_token(payload: Oauth2AccessTokenRequest) -> Result<Oauth2AccessTokenResponse> {
     debug!("Creating access token for {:?}", payload.platform);
 
-    let client =
-        get_client(payload.platform)?.set_redirect_uri(RedirectUrl::new(payload.redirect_uri)?);
-
-    let token = client
-        .exchange_code(AuthorizationCode::new(payload.auth_code))
-        .request_async(async_http_client)
-        .await?;
-
-    // Convert response to the POD like
-    Ok(Oauth2AccessTokenResponse {
-        access_token: token.access_token().secret().clone(),
-        token_type: token.token_type().as_ref().to_string(),
-        refresh_token: token.refresh_token().map(|tok| tok.secret().clone()),
-        expires_in: token.expires_in().map(|d| d.as_secs()),
-    })
+    match payload.platform {
+        Platforms::Gitlab => {
+            let token = exchange_code::<GitlabExtras>(payload).await?;
+
+            Ok(Oauth2AccessTokenResponse {
+                access_token: token.access_token().secret().clone(),
+                token_type: token.token_type().as_ref().to_string(),
+                refresh_token: token.refresh_token().map(|tok| tok.secret().clone()),
+                expires_in: token.expires_in().map(|d| d.as_secs()),
+                created_at: Some(token.extra_fields().created_at),
+            })
+        }
+    }
 }
 
 pub async fn refresh_token(
@@ -53,21 +77,19 @@ pub async fn refresh_token(
 ) -> Result<Oauth2AccessTokenResponse> {
     debug!("Refreshing token for {:?}", payload.platform);
 
-    let client =
-        get_client(payload.platform)?.set_redirect_uri(RedirectUrl::new(payload.redirect_uri)?);
-
-    let token = client
-        .exchange_refresh_token(&RefreshToken::new(payload.refresh_token))
-        .request_async(async_http_client)
-        .await?;
-
-    // Convert response to the POD like
-    Ok(Oauth2AccessTokenResponse {
-        access_token: token.access_token().secret().clone(),
-        token_type: token.token_type().as_ref().to_string(),
-        refresh_token: token.refresh_token().map(|tok| tok.secret().clone()),
-        expires_in: token.expires_in().map(|d| d.as_secs()),
-    })
+    match payload.platform {
+        Platforms::Gitlab => {
+            let token = exchange_refresh_token::<GitlabExtras>(payload).await?;
+
+            Ok(Oauth2AccessTokenResponse {
+                access_token: token.access_token().secret().clone(),
+                token_type: token.token_type().as_ref().to_string(),
+                refresh_token: token.refresh_token().map(|tok| tok.secret().clone()),
+                expires_in: token.expires_in().map(|d| d.as_secs()),
+                created_at: Some(token.extra_fields().created_at),
+            })
+        }
+    }
 }
 
 struct PlatformConfiguration {
@@ -89,7 +111,36 @@ lazy_static! {
     )]);
 }
 
-fn get_client(platform: Platforms) -> Result<BasicClient> {
+async fn exchange_refresh_token<PlatformExtras: ExtraTokenFields>(
+    payload: Oauth2RefreshTokenRequest,
+) -> Result<CustomTokenResponse<PlatformExtras>> {
+    let client = get_client::<PlatformExtras>(payload.platform)?
+        .set_redirect_uri(RedirectUrl::new(payload.redirect_uri)?);
+
+    Ok(client
+        .exchange_refresh_token(&RefreshToken::new(payload.refresh_token))
+        .request_async(async_http_client)
+        .await?)
+}
+
+async fn exchange_code<PlatformExtras: ExtraTokenFields>(
+    payload: Oauth2AccessTokenRequest,
+) -> Result<CustomTokenResponse<PlatformExtras>> {
+    let client =
+        get_client(payload.platform)?.set_redirect_uri(RedirectUrl::new(payload.redirect_uri)?);
+
+    Ok(client
+        .exchange_code(AuthorizationCode::new(payload.auth_code))
+        .request_async(async_http_client)
+        .await?)
+}
+
+/// Creates oauth2 client for specific platform
+/// Client defines exact type of token response, it might be different per platform
+/// The details are defined by PlatformExtras template parameter
+fn get_client<PlatformExtras: ExtraTokenFields>(
+    platform: Platforms,
+) -> Result<CustomClient<PlatformExtras>> {
     let config = PLATFORM_CONFIG.get(&platform).ok_or(Error {
         code: StatusCode::INTERNAL_SERVER_ERROR,
         msg: format!("Missing configuration for {platform:?}"),
@@ -103,7 +154,7 @@ fn get_client(platform: Platforms) -> Result<BasicClient> {
             .context(|| format!("missing: {}", config.client_secret_env))?,
     );
 
-    Ok(BasicClient::new(
+    Ok(CustomClient::<PlatformExtras>::new(
         client_id,
         Some(client_secret),
         config.auth_url.clone(),