dgate performance and regen

    Hi,


    Conquest's data storage is on a RAID storage (mdadm) and conquest data stored is now at 4.6 TB out of 12 TB space. Conquest which was performing fine before, now is performing very slow in storing and retrieving data. RAID is just fine and I get pretty good write/read speeds via dd tests. I tried to regen from stored data. I issued "dgate -v -r > /tmp/log 2>&1" on last Friday evening at 4:30 PM and it was still running at 1:40 PM on Tuesday (today). It would be nice if Linux version of dgate regen shows % completed in verbose mode. I aborted the regen and restored conquest database from /var/lib/mysql backup that I did before the regen. I am still faced with slow performance.


    I would like to know what causes slowness in dgate's storing and retrieving. Also I would like to know how to make the regen process faster. I guess I can also do simply "./dgate -r" that is without verbose. Machine has 8-Core 3.6 GHz CPU and 16GB RAM. Other processes such as storescp etc., works faster on the same machine.


    I would also like know if there any mysql specific configuration that people are using with conquest.


    OS: 3.9.10-100.fc17.x86_64


    ./dgate -v
    DGATE (1.4.17alpha, build Mon Dec 3 10:54:01 2012, bits 64) is running as threaded server
    Database type: native MySQL connection


    I can also take this to dgate 1.4.17d if that helps.


    Please let me know. Thanks.

    Hi


    Regen speed depends on the read speed of the RAID where the DICOM data is stored and on the database type and speed of the driver the DB is on.


    Though I use windows my regen speeds with CT/MRI images where between 230-400 images/s (average around 300)- depending on image size and or if it was a continuous or more random read.


    DB: MySQL
    DB drive: Intel SSD
    32GB RAM


    I can't think of any reason why write speed of Conquest would drop using Linux and MADM with so much free space left. So I am pretty sure it is related to DB performance.


    I can pull my MySQL setting on the weekend and pot them here. Otherwise you can do some research on the internet regarding optimal MySQL settings - isn't that many settings that are needed

    Hi,


    I would guess it is database performance as well. You can try to select a NULL database for testing (see manual) writing. That would single out the effect of a database store on speed.


    Marcel

    I did only change the basic stuff:


    [code][Basic settings


    Here are 3 settings that you should always look at. If you do not, you are very likely to run into problems very quickly.


    innodb_buffer_pool_size: this is the #1 setting to look at for any installation using InnoDB. The buffer pool is where data and indexes are cached: having it as large as possible will ensure you use memory and not disks for most read operations. Typical values are 5-6GB (8GB RAM), 20-25GB (32GB RAM), 100-120GB (128GB RAM).


    innodb_log_file_size: this is the size of the redo logs. The redo logs are used to make sure writes are fast and durable and also during crash recovery. Up to MySQL 5.1, it was hard to adjust, as you wanted both large redo logs for good performance and small redo logs for fast crash recovery. Fortunately crash recovery performance has improved a lot since MySQL 5.5 so you can now have good write performance and fast crash recovery. Until MySQL 5.5 the total redo log size was limited to 4GB (the default is to have 2 log files). This has been lifted in MySQL 5.6.


    Starting with innodb_log_file_size = 512M (giving 1GB of redo logs) should give you plenty of room for writes. If you know your application is write-intensive and you are using MySQL 5.6, you can start with innodb_log_file_size = 4G.


    max_connections: if you are often facing the ‘Too many connections’ error, max_connections is too low. It is very frequent that because the application does not close connections to the database correctly, you need much more than the default 151 connections. The main drawback of high values for max_connections (like 1000 or more) is that the server will become unresponsive if for any reason it has to run 1000 or more active transactions. Using a connection pool at the application level or a thread pool at the MySQL level can help here./code]

    Thank you both.


    Before making any changes to MySQL database, I was getting only 4 MR images per second in storage. After database setting changes as shown below, it only improved to 8 MR images per second.
    innodb_buffer_pool_size=10G
    innodb_log_file_size = 512M
    max_connections was not an issue.


    Clearly, regen was not an option and OS/database issue had to be dealt with. Then on a test machine installed RHEL7.2 with mariadb, imported conquest database, compiled and installed dgate 1417d version. Operating system disk was a SSD. This setup gave speed of 65 MR images per second in storage. With this result, I switched operating system from FedoraCore17 to RHEL7.2. I also replaced spinning disk with SSD for OS disk. Now I am getting 65 images per second in storage, and I do have the above database settings in /etc/my.cnf.


    Thanks again!

    Hi


    Glad to hear!


    Your HDD speeds where subterranean - with decent HDD 10k rpm for DB and OS with Conquest on another drive the speed should have been around 20-40 images/s - so not too far from your speed with SSD now as at a certain point network speed (small files) and overhead accumulates.


    With SSD you could try sending a large study from 2 clients at once - if your RAID is fast speed will still go up...

    OS and Mysql were sitting on 7200 rpm SATA drive. The dd read/write speeds were not the issue on this disk; I was getting ~160+ MB/s write speeds, and this should have yielded much more than 8 MR images per second; though I was writing to mdadm array. It had to do with FC17/mysql related issue. Instead of spending too much time troubleshooting the cause of slowness on an unsupported OS, I figured trying on latest stable platform would be the best and fastest option. Conquest data itself resides on mdadm as mentioned above. I just took the OS install (upgrade) opportunity to pull out spinning disk and replace with SSD.


    Now performance is not an issue. Thanks.

    Write speed in MB/s is not relevant for DB for that IOPs is mor einteresting and those are low for 7.2K driver so DB usually need 10 or 15K drives. But since all is good now there is no need to argue about that :-)

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!

Register Yourself Login