Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • telegram telegram
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 0
    • Issues 0
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 1
    • Merge requests 1
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • MemriMemri
  • plugins
  • telegramtelegram
  • Merge requests
  • !2

Optimizations

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Aziz Berkay Yesilyurt requested to merge aziz/optim into dev Nov 04, 2022
  • Overview 2
  • Commits 3
  • Pipelines 1
  • Changes 4

Some optimizations to speed up importing messages. Speed up is 2x, from 400 sec to 200 sec to import 10k messages, on local pod.

General idea is to:

  • Use set instead of list when we check for existance of items.
    • This requires that Item is hashable, which is not. So I patched schema items with __hash__ function and used external_ids as hashes, which should be unique anyway.
  • Use lru_cache when querying item by id, so we don't have to wait for pod to respond.
Edited Nov 07, 2022 by Aziz Berkay Yesilyurt
Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: aziz/optim