repeated login to SQL

  • Hello, I'm at the stage of porting the data from old (SQL2005+conquest 1.4...) to new (SQL 2019+conquest 1.5) platform, and I found a stange behaviour that apparently hampers the performences with MsSQL.


    If I transfer the images from another instance, or put them in the "incoming" folder, the current setup imports around 50 img/s. However, I noticed that the hardware is barely used: processsors around 20%, processor queue 0, disk queue 0 (see attachments, the hardware is old but capable: 2x 4 cores xeon, SAN based on 15k disks). I noticed that during import the lsass process uses as many as resources as sql+conquest together. I investigated a bit the issue, and I found that dgate64 logins and logout from SQL for each image. Apparently, the vast majority of time is used to do these continuos logins, that seem to tax lsass and cause unneded delays. I wonder if these logins for each image are needed. Couldn't be kept logged-in the proces, at leas for some minutes?


    Interestingly, if the database is regenerated (dgate64 -ARMAG0) a single login is performed and all the images are imported under the same connection to SQL, at an incommensurably higher speed. I understand that in this case no disk write is implied, but it looks like it is not the disk the bottleneck. This is also suggested by the fact that remote transfer (implying full disk write) and data in "incoming" (implying only a move between directories in the same partition) hve similar speeds of 40-60 img/s.


    BTW the documentation looks to suggest that pooled logins are faster tahn unpooled ones, and teh logins are of the unpooled variety.

    Best, Federico

  • Hi Federico,


    Indeed conquest connects often to the database. The code to connect to the database is in ODBCI.cpp. If you can show me how to make the login pooled, I can implement that. Pooling on application level is a bit harder because many actions can occur at the same time.


    Marcel

  • Hi Federico,


    It is possible to reduce such re-logins, but at a risk of breaking reliability. Are you willing to test intermediate versions later? And let me know which operations you would like to have accelerated.


    Marcel

  • Hi Marcel, sorry for m late reply, I missed your post.


    Yes, I'm wlling to test any beta relaease you find useful.


    In my setup the most offending operations are plain transfers (eg from the scanner, or from another conquest instance). The transfer is fast enough to keep up with two scanners working simultaneously in high troughput mode (multiband fMRI etc.), but no room for problems creating a backlog, or for more scanner. It is odd considering that less than 20% of resources are used.


    I played with the various ODBC drivers, and the problem (ie a new SQL login for each image) is there with any driver supporting pooling.


    Best, federico

  • Hi,


    if i use the server GUI to copy a patient (112 objects) from conquest to conquest I see 7 new database opens in total, whch is for sending and recieving together. You can check this with this command:


    dgate64 --lua:return(Global.DatabaseOpen)


    Which uses one database open itself. This prints is a counter on the internal database driver.


    So I agree that loading data from incoming is inefficient with database connections, plain transfers are however not. Can you check again and indicate which operations need attention.


    regards,


    Marcel

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!