Some optimizations to speed up importing messages. Speed up is 2x, from 400 sec to 200 sec to import 10k messages, on local pod.
General idea is to:
set
instead of list
when we check for existance of items.
Item
is hashable, which is not. So I patched schema items with __hash__
function and used external_id
s as hashes, which should be unique anyway.lru_cache
when querying item by id, so we don't have to wait for pod to respond.