Dataimport handler for Sunburnt Solr python library

Dataimport handler

I had to trigger the dataimporter delta-import command from the code so I added support for this function.

Example:

from sunburnt import SolrInterface
SOLR_URL = 'http://some.url:8983/solr/'
_s = SolrInterface(SOLR_URL)
_s.dataimport('delta-import')
_response = _s.dataimport('status')
print _response.import_status
print _response.import_status.time_taken

from sunburnt import SolrInterface

SOLR_URL = 'http://some.url:8983/solr/'

_s = SolrInterface(SOLR_URL)

_s.dataimport('delta-import')

_response = _s.dataimport('status')

print _response.import_status

print _response.import_status.time_taken

Following properties are available in the SolrImportStatus (_response.import_status) object:

    markers = [
        ('Total Requests made to DataSource', 'requests'),
        ('Total Rows Fetched', 'rows_fetched'),
        ('Total Documents Skipped', 'docs_skipped'),
        ('Full Dump Started', 'full_started'),
        ('Delta Dump started', 'delta_started'),
        ('Committed', 'committed'),
        ('Optimized', 'optimized'),
        ('Total Changed Documents', 'docs_changed'),
        ('Total Documents Processed', 'docs_processed'),
        ('Time taken ', 'time_taken'),
    ]

markers = [

('Total Requests made to DataSource', 'requests'),

('Total Rows Fetched', 'rows_fetched'),

('Total Documents Skipped', 'docs_skipped'),

('Full Dump Started', 'full_started'),

('Delta Dump started', 'delta_started'),

('Committed', 'committed'),

('Optimized', 'optimized'),

('Total Changed Documents', 'docs_changed'),

('Total Documents Processed', 'docs_processed'),

('Time taken ', 'time_taken'),

]

Source code can be found on my github page: https://github.com/charlesnagy/sunburnt

You might like these too

Python timeit – when speed matters – S... Although there are always multiple ways to solve a single problem not always the most elegant is the best performant. Python gives a perfect tool to c...

Priority queue in Redis aka ZPOP Redis list is great to use it as a processing queue. So great that it sort of became the standard and many people using that instead of dedicated queu...

5 things to love about python Syntax Takes a bit of time to get used to it but after a certain point it just becomes natural. Like english. No heavy-weight operators or complicate...

About charlesnagy

I'm out of many things mostly automation expert, database specialist, system engineer and software architect with passion towards data, searching it, analyze it, learn from it. I learn by experimenting and this blog is a result of these experiments and some other random thought I have time to time.

View all posts by charlesnagy →

Dataimport handler for Sunburnt Solr python library

Dataimport handler

You might like these too

About charlesnagy

Categories

About Charles Nagy

Community

Tweets

Links