Python MySQLdb vs mysql-connector query performance

Query times for random PK using MySQLdb and mysql-connector

There are a lot of python driver available for MySQL and two stand out the most. The one, traditionally everybody’s choice, sort of industrial standard MySQLdb. It uses a C module to link to MySQL’s client library. Oracle’s mysql-connector on the other hand is pure python so no MySQL libraries and … Continue reading

Python timeit – when speed matters – SQL IN query with cursor.execute

Although there are always multiple ways to solve a single problem not always the most elegant is the best performant. Python gives a perfect tool to check primitives (or even a bit more complex) structure’s speed. This comes really handy when trying to figure out of a loop or a … Continue reading

Stream cypher (encrypt) with digital envelope in Python

Generating. storing and keeping inventory of hundreds of terabyte large database backups is a challenge by itself which we do on daily basis. It’s even a bigger challenge to store them encrypted which is the requirement sometimes. The backup data is being pushed through a socket generated by percona-xtrabackup. On … Continue reading

Useful python – Class based decorators and context managers

useful-python

There are many things why Pyhton is my standard go-to language if it comes to implement something. It’s either a website, automation, data-mining or complex calculation Python excels in most of it. I decided to write some of my favourite things which just makes things cleaner and easier to implement.

Continue reading

Solr Stats component is available in sunburnt

Stats

StatsComponent is now available with stats function in the suburnt Solr python client library.

More info: http://wiki.apache.org/solr/StatsComponent

Example:

For the source code go to my github repo: https://github.com/charlesnagy/sunburnt

Flashcache stats collector for Diamond / Graphite

I have upgraded the LVM collector in my StorageCollector repository on github (https://github.com/charlesnagy/Diamond-StorageCollectors) to match the new directory structure of Diamond and added support to collect flashcache metrics.

Installation

Move the flashcache directory to your collectors directory which should exist on /usr/local/share/diamond/collectors or /usr/share/diamond/collectors path depending on your installation.

Continue reading

Dataimport handler for Sunburnt Solr python library

Dataimport handler

I had to trigger the dataimporter delta-import command from the code so I added support for this function.

Example:

Following properties are available in the SolrImportStatus (_response.import_status) object:

Source code can be found on my github page:  Continue reading