--- title: Title keywords: fastai sidebar: home_sidebar nb_path: "nbs/TwitterImporter.ipynb" ---
{% raw %} 
Could no connect to backend
Running twitter importer
Preparing to fetch user data...
Successfully fetched user details.
Saving to pod...
Fetching followers...
HTTPConnectionPool(host='localhost', port=3030): Max retries exceeded with url: /v3/5321525286271939638329683169641785506738848570287738248673946278/create_edge (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffc19a79f70>: Failed to establish a new connection: [Errno 61] Connection refused'))
Fetching followings...
HTTPConnectionPool(host='localhost', port=3030): Max retries exceeded with url: /v3/5321525286271939638329683169641785506738848570287738248673946278/create_edge (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffc19a90e80>: Failed to establish a new connection: [Errno 61] Connection refused'))
Fetching tweets...
Successfully fetched tweets
HTTPConnectionPool(host='localhost', port=3030): Max retries exceeded with url: /v3/5321525286271939638329683169641785506738848570287738248673946278/create_edge (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffc19a6aa30>: Failed to establish a new connection: [Errno 61] Connection refused'))
Successfully saved tweets to pod
{% endraw %} {% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %}

class TwitterImporter[source]

TwitterImporter(access_token, access_secret, consumer_key, consumer_secret, user_handle)

A class to represent a TwitterImporter.

...

Attributes

access_token: str A valid access_token for a given twitter developer access_secret: str A valid access_secret for a given twitter developer consumer_key: str A valid consumer_key for a given twitter developer consumer_key: str A valid consumer_key for a given twitter developer user_handle: str A twitter user handle from which to import data

Methods

authenticate_user(limit=None): Returns an api class with methods to access twitter endpoints. get_user: Returns user details import_followers(limit=None): Returns the followers of a given user handle. import_followings(limit=None): Returns the followings of a given user handle. import_timeline(limit=None): Returns the user_timeline list of a given user handle.

{% endraw %}

Full importer run method

{% raw %}
{% endraw %} {% raw %}

run[source]

run(client, tokens, username, max_amount)

Imports user, friends, followers and timeline into the pod client.

        Parameters:
                pod_client (object): An instance of the PodClient client.
                tokens (object): An object with twitter tokens to access the API.
                username (str): A string for the user from whom to fetch data.

        Returns:
                followers (dict): An object of followers.
{% endraw %} Twitter client. This is where memri auth credentials are passed

Add your authentication credentials from your twitter developer account

Client credentials:

App Key === API Key === Consumer API Key === Consumer Key === Customer Key === oauth_consumer_key
App Key Secret === API Secret Key === Consumer Secret === Consumer Key === Customer Key === oauth_consumer_secret
Callback URL === oauth_callback

Token credentials:

Access token === Token === resulting oauth_token
Access token secret === Token Secret === resulting oauth_token_secret

Set up importer instance, with the token and user handle

{% raw %}
{% endraw %}

Instatiate the pod

{% raw %}
{% endraw %} {% raw %} 
client.test_connection()

Succesfully connected to pod

True
{% endraw %}

Add data schema to pod

{% raw %} 
client = PodClient()
# Add schemas

user_item = User(1, "name", "screen_name", "location", "profile_location", "description", "url", "entities", "protected", 1, 1, 1, "created_at", 1, "utc_offset", "time_zone", "geo_enabled", "False", 30, "lang", "status", "False", "False", "True", "Yellow", "profile_background_image_url", "profile_background_image_url_https", "profile_background_tile", "profile_image_url", "profile_image_url_https", "profile_link_color", "profile_sidebar_border_color", "profile_sidebar_fill_color", "profile_text_color", "True", "False", "True", "False", "True", "True", "True", "translator_type", "withheld_in_countries")
client.add_to_schema(user_item);

tweet_item = Tweet.from_data("created_at", 1363430961812283392, 835191402, "full_text", "False", "displayRextRange", "entities", "source", "None", "None", "None", "None", "None", "user", "None", "None", "None", "None", "False", 20, 10, "True", "True", "lang")
client.add_to_schema(tweet_item);

contact = Contact(1335859528886349824, 835191402, "name", "screenName", "location","description", "url", "entities", "protected", 8, 9, 0, "twitterDateCreated", 1, "utcOffset","timeZone","True", "False", 6, "lang", "status", "contributorsEnabled", "isTranslator","isTranslationEnabled", "profileBackgroundColor","profileBackgroundImageUrl", "profileBackgroundImageUrlHttps","profileBackgroundTile", "profileImageUrl", "profileImageUrlHttps", "profileLinkColor","profileSidebarBorderColor", "profileSidebarFillColor", "profileTextColor", "profileUseBackgroundImage", "hasExtendedProfile","defaultProfile", "defaultProfileImage", "following", "liveFollowing","followRequestSent", "notifications", "muting", "blocking", "blockedBy","translatorType", "withheldInCountries", "Follower")
client.add_to_schema(contact);
{% endraw %}

Run importer. Include your preferred username e.g "elonmusk"

{% raw %} 
Running twitter importer
Preparing to fetch user data...
Successfully fetched user details.
Saving to pod...
User (#a8742da7cfa6f6a593ee02d4a6c47f00) with twitterId  835191402 added
Fetching followers...
Contact (#cccfa6b6c39269b78ddcaa5fb170e85d) with twitterId  1335859528886349824  added
Fetching followings...
Contact (#d27dc9ad87be8bb2b81517f11d5edcf6) with twitterId  330996760  added
Fetching tweets...
Successfully fetched tweets
Tweet (#ab373cca89214848c6a2521400f9bac4) with twitterId  1393544220196098050 updated
Successfully saved tweets to pod
{% endraw %} {% raw %}
{% endraw %}