# Exporting Logs With Python

Key Point

How to use Python to export Onna audit logs as JSON and save them to /tmp.

💡 5 min read


Disclaimer

This guide is written for Linux and macOS, depending on your Operating System you may need to adjust.

This guide is based on Python 3.7 and uses the hosted Onna SaaS platform.

# Requirements

Note

Make sure that the version of Python is the same as used with the Onna instance.

The hosted platforms (e.g https://$COMPANY.onna.io or https://enterprise.onna.com/$COMPANY) are using Python 3.7.4.

If Rust is more to your liking check out onna-logs

# Prerequisite

The activitylog endpoint requires specific permissions.

If you want to run this script unattended, like as a cron job, the user account used to authenticate must be a Service Account.

Please contact Onna support (support [at] onna [dot] com) for assistance with setting up a Service Account.

Note

To be able to export data from your Onna instance, you'll need the following credentials:

USERNAME: Service Account or username
PASSWORD: password
ACCOUNT: the Onna account name
ACCOUNT_URL: the URL of your account, e.g https://$COMPANY.onna.io or https://enterprise.onna.com/$COMPANY

# Setup

Create a directory for the script and its dependencies

mkdir $DIR #choose a name for the directory
1

Change into the newly created directory

cd $DIR
1

Create a file with the name requirements.txt with the following content

certifi==2019.11.28
chardet==3.0.4
idna==2.8
python-dateutil==2.8.1
requests==2.22.0
six==1.14.0
urllib3==1.25.8
1
2
3
4
5
6
7

In the same directory create a file with the name auditlogs.py with the following content

import argparse
import json
import sys
import requests


from dateutil.parser import parse


argparser = argparse.ArgumentParser(
    description="Download Onna audit logs",
    epilog="And write to a file in the /tmp directory",
)
argparser.add_argument(
    "--username", required=True, type=str, help="Service Account username"
)
argparser.add_argument("--password", required=True, type=str, help="password")
argparser.add_argument(
    "--account", required=True, type=str, help="the Onna account name"
)
argparser.add_argument(
    "--account_url",
    required=True,
    type=str,
    help="the URL of your account, e.g https://company.onna.io or https://enterprise.onna.com",
)
argparser.add_argument(
    "--from_date",
    required=True,
    type=str,
    help="start date. most date formats are accepted",
)
argparser.add_argument(
    "--to_date",
    required=True,
    type=str,
    help="end date. most date formats are accepted",
)
argparser.add_argument("--fname", required=True, type=str, help="name of the file")
argparser.add_argument(
    "--container", required=False, default="rel0", help="name of the account container"
)
argparser.add_argument(
    "--size", required=True, default="10", type=str, help="result size"
)


def auth_code(url=None):
    if not url:
        raise Exception
    resp = requests.get(url)
    if resp.status_code == 200:
        return resp.json()["auth_code"]


def auth_token(auth_code, username, password, scope, base_url):
    payload = {
        "grant_type": "user",
        "code": auth_code,
        "username": username,
        "password": password,
        "scopes": [scope],
        "client_id": "canonical",
    }
    headers = {"Accept": "application/json"}
    resp = requests.post(
        f"{base_url}/auth/oauth/get_auth_token",
        headers=headers,
        data=json.dumps(payload),
    )
    if resp.status_code == 200:
        jwt_token = resp.text
    return jwt_token


def activity_log(
    dt_from, dt_to, size, jwt_token, base_url, account, container, cursor=None
):
    headers = {"Accept": "application/json", "Authorization": f"Bearer {jwt_token}"}
    url = f"{base_url}/api/{container}/{account}/@activityLog?epoch_from={dt_from}&epoch_to={dt_to}&size={size}"
    if cursor is not None:
        url = f"{base_url}/api/{container}/{account}/@activityLog?epoch_from={dt_from}&epoch_to={dt_to}&size={size}&cursor={cursor}"

    resp = requests.get(url, headers=headers)
    if resp.status_code == 200:
        return resp


def write_log(fname, resp):
    fname = f"/tmp/{fname}.json"
    # write the file to /tmp/
    with open(fname, "w") as f:
        for chunk in resp:
            json.dump(chunk, f)


def main():
    try:
        from_date = int(parse(args.from_date).timestamp())
    except (TypeError, ValueError):
        sys.exit(1)
    try:
        to_date = int(parse(args.to_date).timestamp())
    except (TypeError, ValueError):
        sys.exit(1)

    username = args.username
    password = args.password
    account = args.account
    base_url = args.account_url
    container = args.container
    fname = args.fname
    size = args.size

    auth_code_url = f"{base_url}/api/{container}/{account}/@oauthgetcode?client_id=canonical&scope={account}"
    code = auth_code(auth_code_url)
    token = auth_token(code, username, password, account, base_url)

    result = activity_log(from_date, to_date, size, token, base_url, account, container)
    cursor = result.json().get("cursor", None)
    user_activity_response = list()
    user_activity_response = result.json()["items"]
    if cursor is not None:
        i = 0
        while cursor is not None:
            result = activity_log(
                from_date, to_date, size, token, base_url, account, container, cursor
            )
            user_activity_response.append(result.json())
            cursor = result.json().get("cursor", None)
            i += 1

    fname = f"{fname}_user_activity_{from_date}-{to_date}"
    write_log(fname, user_activity_response)


if __name__ == "__main__":
    args = argparser.parse_args()
    main()

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140

Still in the same directory, create a virtual environment

python3 -m venv env
1

Activate the virtualenv

source env/bin/activate
1

Install all dependencies with pip

pip install -r requirements.txt
1

# Script

# Usage

Note

Since you are running the script in your activated Python 3 virtual environment python is linked to Python 3.

Run python auditlogs.py --help to get an overview.

# Syntax

python auditlogs.py --username $USERNAME --password $PASSWORD --account $ACCOUNT \
--account_url $ACCOUNT_URL \
--from_date $FROM_DATE --to_date $TO_DATE \
--fname $FNAME [--container $CONTAINER] \
--size $SIZE
1
2
3
4
5

Note

[--container $CONTAINER] refers to the container holding all your data in the Onna instance.

Usually the name is the same as your account name and [--container $CONTAINER] is not needed.

Please contact Onna support (support [at] onna [dot] com) if you need assistance.

# Example


python auditlogs.py --username example1 --password 1234 --account example \
--account_url https://enterprise.onna.com \
--from_date "01/01/2020" --to_date "01/20/2020" \
--fname example-audit --container example \
--size=10
1
2
3
4
5
6

Running the script will do the following:

It will export all audit logs from the 01-01-2020 till the 01-20-2020 of the account example running on https://enterprise.onna.com to /tmp/example-audit.json.

The script uses the username example1 with the password 1234.

Last Updated: 6/18/2020, 1:33:36 PM