Posts by manzing

  • Conquest 1.5.0b with MS SQL issue

    Quote from marcelvanherk

    Ah,


    I have an idea. The query probably asks for the number of study related instances, which is queried every time, for every study. If you disable enableComputedFields the query will be loads faster.


    Marcel

    WOW, Marcel, thank you so much, you just gave me THE solution.

    No more excess login request, no more timeout, my client was able to display the 62476 results to my big 10 years query in 94 seconds.

    I am going to write this down right now : never enable "enableComputedFields" when using MS SQL server DB.


    Many thanks you are definitly my Hero :)

  • Conquest 1.5.0b with MS SQL issue

    Hi Marcel,


    Yes I have 2 lua scripts called in the lua section of dicom.ini that I use to allow multiple modalities query but actually it does not work anymore as it used to.

    I disabled it and tried a search, the behavior is the same : more than 20mn to process the query.

    My DB is rather large though, nearly 6M images and 4+TB data.

  • Conquest 1.5.0b with MS SQL issue

    Marcel,

    Here is a partial log after a 10 years archive query :


    After about 30 secondes (less than a minute for sure), my dicom client ends search and displays : 0 results.

    As you can see in the log, the query took 24 minutes to complete !

    I have about 40 to 50 lines "logging in to SQL" each second during the whole process (and about 1-2 / second when idle).

    This should obviously explain the issues I faced since I use SQL.

  • Conquest 1.5.0b with MS SQL issue

    Hi Marcel,


    Sadly I have no relevant entries in SQL logs.

    I tried to raise timeout duration, disabled the query governor with no change : results to big queries remain 0, and resources are still eaten for a while.

  • Conquest 1.5.0b with MS SQL issue

    CPU usage is just an indication (all processes togheter use about 5-10% this is quite low).

    The real issue is that after a query time out, dgate, SQL and lsass continue to consume resources more than normal for a long time, maybe 10 to 30mn depending on queries.

    And this combines : if several "big" queries comes every 2 minutes for exemple, ram usage by dgate grows exponentially, as the CPU for other processes does beecause each query add a little overload.

  • Conquest 1.5.0b with MS SQL issue

    Marcel,


    I've run the diag 3 times : before query, during (big) query, after query showed 0 results in client.

    A lot more threads than yours, don't know if that matters.


    Code: Before Query
    C:\Windows\system32>C:\PACS\dgate64 --display_status:

    DICOM server 'CONQUEST' (version 1.5.0-alpha-test1, port 104, bits 64) was started on Mon Apr 12 09:10:39 2021

    Old JPEG decoder=0, JPEGLIB jpeg codec=1, LIBJASPER jpeg2000 codec=1

    Run time (s) total 15022, query 0, load 51, save 8, compress 6, process 0, gpps 9

    Associations=29462; Threads=28169 (1 open); Images sent=11910, recieved=2663, forwarded=0

    Images printed=0, in color=0

    Activity: Echo:4, Find:26410, Move:381, Unknown:0, gpps:5396794

    Images (de)compressed: NKI 0, JPEG 115, JPEG2000 0, RLE 0, Planes removed 0, Palettes removed 0, Downsize 0

    Space on MAG0  : 6818394 MByte

    Database type: ODBC connection
    Code: During query
    C:\Windows\system32>C:\PACS\dgate64 --display_status:

    DICOM server 'CONQUEST' (version 1.5.0-alpha-test1, port 104, bits 64) was started on Mon Apr 12 09:10:39 2021

    Old JPEG decoder=0, JPEGLIB jpeg codec=1, LIBJASPER jpeg2000 codec=1

    Run time (s) total 15027, query 0, load 51, save 8, compress 6, process 0, gpps 10

    Associations=29503; Threads=28211 (2 open); Images sent=11910, recieved=2672, forwarded=0

    Images printed=0, in color=0

    Activity: Echo:5, Find:26440, Move:381, Unknown:0, gpps:5409117

    Images (de)compressed: NKI 0, JPEG 115, JPEG2000 0, RLE 0, Planes removed 0, Palettes removed 0, Downsize 0

    Space on MAG0  : 6818376 MByte

    Database type: ODBC connection
    Code: 2mn after 0 results search
    C:\Windows\system32>C:\PACS\dgate64 --display_status:

    DICOM server 'CONQUEST' (version 1.5.0-alpha-test1, port 104, bits 64) was started on Mon Apr 12 09:10:39 2021

    Old JPEG decoder=0, JPEGLIB jpeg codec=1, LIBJASPER jpeg2000 codec=1

    Run time (s) total 15038, query 0, load 51, save 8, compress 6, process 0, gpps 10

    Associations=29578; Threads=28291 (2 open); Images sent=11910, recieved=2686, forwarded=0

    Images printed=0, in color=0

    Activity: Echo:5, Find:26501, Move:381, Unknown:0, gpps:5472920

    Images (de)compressed: NKI 0, JPEG 115, JPEG2000 0, RLE 0, Planes removed 0, Palettes removed 0, Downsize 0

    Space on MAG0  : 6818355 MByte

    Database type: ODBC connection

    You will notice that version is 1.5.0 alpha, I tried this version after similar issue on 1.5.0b, I guess I forgot to revert to a "stable" version.

    I just updated and am back to 1.5.0b now.


    Code: After update to 1.5.0b
    C:\Windows\system32>C:\PACS\dgate64 --display_status:

    DICOM server 'CONQUEST' (version 1.5.0b, port 104, bits 64) was started on Wed Apr 14 12:56:43 2021

    Old JPEG decoder=0, JPEGLIB jpeg codec=1, LIBJASPER jpeg2000 codec=1

    Run time (s) total 29, query 0, load 0, save 0, compress 0, process 0, gpps 0

    Associations=375; Threads=393 (1 open); Images sent=3, recieved=9, forwarded=0

    Images printed=0, in color=0

    Activity: Echo:0, Find:364, Move:2, Unknown:0, gpps:44400

    Images (de)compressed: NKI 0, JPEG 0, JPEG2000 0, RLE 0, Planes removed 0, Palettes removed 0, Downsize 0

    Space on MAG0  : 6818232 MByte

    Database type: ODBC connection

  • Conquest 1.5.0b with MS SQL issue

    I've had no issue on sqlite but database is way smaller as there are only CT.

    An empty query does not seem to use additional memory.


    I just tested a "big" query on MS SQL pacs (10 years), and Microdicom took about 1mn to show 0 results (instead of thousands).

    The issue is still there because since this moment, idle dgate process uses 0.5% CPU (usually, idle is between 0 to 0.1%), MS SQL process continue to use more CPU than usual, and also lsass.exe which always consume CPU each time the problem occures.


    In fact I don't understand why I get 0 results, neither why resources continue to be used after the query is apparently over.

    I am pretty sure the issue is related to big queries, but only MS SQL DB seems to be affected.

    Maybe my SQL Server needs some tuning, but I've already tried several tweaks with no results.


    Is there a way to force stop queries, let's say if results are > 100 ie?


    As I write I sent a new big query : dgate64 is now 0.9% CPU, lsass 2.6% and SQL 3.5% which is almost twice bigger, that looks like the previous query is stuck in background...

    Not sure I am clear, sorry.

  • Conquest 1.5.0b with MS SQL issue

    Hi,


    I had some issues again last month, something like a memory leak, and dgate process that becomes so slow so I have to restart it.

    I found some "serious" lead as I was making some testing with an Orthanc server : empty queries make dgate process grows and even after a dicom client reports "no results" the process continues to grow for a while.

    If several clients do the same thing (especially with this awful settings "auto refresh" results in Horos / Osririx search) it could lead to potential instability.

    I have many clients, and a lot of them used by students who can mess with results "auto refresh" or empty queries.

    The issue does not occur making an empty query in the internal search module of Conquest GUI (memory usage does not vary in this case).


    So, I added this line in the lua section of dicom.ini :

    Code
    QueryConverter0 = if (Data.PatientID=="") and (Data.StudyDate=="") and (Data.PatientName=="") then script('destroy') end

    For now (after 2 weeks), I didn't have to restart dgate process and I feel that it is more stable and I faced no slow-down since this moment.


    I hope this will solve the issue for good.

  • Conquest 1.5.0b with MS SQL issue

    Quote from qcor

    Using 1.5.0c (b + few hotfixes by Marcel) I can confirm that dgate64.exe uses about 200Mb while idle and can EASILY use up to 4Gb of ram (or even more) while 'working' but I've >never< seen it crash. Also the memory usage goes down after a while so it doesn't look like a memory leak.

    Back when I had 1.5.0b it was running 24/7 literally for months with zero issues (under moderate load - 150ish MR/CT per day)


    The difference is I'm using PostgresSQL. Maybe it's a db specific issue?

    DB specific issue is exactly what I think as I never faced such issue before, but again, I used to have one instance for each modality and I decided to put all together on one instance so the duty is a lot heavier.

    Till last week, CT exams are one a specific instance using SQLite DB. No issue at this moment but activity is a bit lighter than usual I guess.


    I will monitor and let you know.


    Quote from marcelvanherk

    What db did you use before?


    Marcel


    I had 7 instances using SQLite on my previous server. On my new server, 1 instance using MS SQL at first (but it crashed a lot) and at this moment I still have MS SQL instance for all but the CT, and one instance for CT using SQlite.

  • Conquest 1.5.0b with MS SQL issue

    Hi Marcel,

    It's a brand new server, with 16 GB RAM. Most of the time, it uses about 40%, considering that MS SQL takes 4-5 GB itself.

    The process does not always crash, but as it grows it becomes so slow that it is unusable and I am forced to restart dgate.

    Resources of the server are barely used.

    Besides, when it does operate correctly, I noticed yesterday that the process took only a few MB of memory when transfering big CT exams, like 40-50 MB memory used to retrieve a 1 GB study.

    For now, I moved CT exams on a new instance using SQLite (seems OK for now), and might do the same thing for MRI for stability purpose if necessary as it seems to be linked with big studies.


    The main change with this new server is the use of MS SQL. Coincidence?

  • Conquest 1.5.0b with MS SQL issue

    Hi Marcel,


    Thanks, I guess I have to either change DB type or / and configure one instance for each modality as it used to be.


    That way it would solve both of these issues, or at least make it a lot easier to manage.

    Put the things all together was a false good idea.

  • Conquest 1.5.0b with MS SQL issue

    Hi Marcel,


    Thanks for your answer.


    1) I didn't check with any other DB as during testing phase everything was OK with MS SQL. The leak occures in production environnement, maybe linked with storing of some modalities as the process seems to grow when receiving images and not free all the RAM after. Now the server is in production and I cannot make those tests without disturbing clinical activity.


    Today I have the opportunity to see a live crash, and the 1.4.19d2 dgate version crashed too. I think this is link to our new scanner which generate huge series. Clinicians sent a 3 Gb study on the PACS, and then tried to retrieve it on several clients simultaneously. dgate uses more CPU and RAM than usual and it grows till about 4 Gb of RAM. It becomes so slow that it is unusable and I had to restart it manually. I've never seen this issue before, when I used one instance per modality, the other difference with my previous server is the MS SQL.

    As the pacs becomes instable, pacstrouble.log gets filled with sql connections errors. This happens with 1.5.0b version (see log below), but NOT with 1.4.19. No errors in log but the server crashes though.


    I believe this is partly due to the use of MS SQL. I am disappointed as indexing is so fast comparing to SQLite I used before, and it seems that I have to go back to SQLite.

    With a new testing instance of 1.5.0b using SQLite, I put the 3Gb scanner exam and downloaded it on my PC in 45s, dgate process took only a few Mb of RAM.

    MS SQL is obviously involved in this issue and I don't know why. I am certainly not the only one to use MS SQL but I can't find any other similar issue in the forum.



    2)

    Looks like Microdicom searches at study level, while Radiant search at image level, that might explain why Microdicom cannot retrieves results for multiple modalities search.

    Reading the forum I understand that modality is defined at series or image level.

    Not sure if there's a way to force an image level search in Microdicom though.

  • Conquest 1.5.0b with MS SQL issue

    Hi all,


    I recently changed my PACS Server and I decided to use a stronger and faster DB (I used integrated SQLite before) so I configured MS SQL Server (2019) on Windows Server 2016 and the latest Conquest PACS 1.5.0 b.

    I quickly faced some big issues.

    Basically, PACS is OK at first but process dgate64.exe start to use more CPU (not so much ,about 5% instead of 0.5% max usually) and more RAM. At some point, memory grows to reach hundreds of Mb or even till 4 Gb and Conquest becomes unusable or simply crash. It got worse and yesterday I had to restart Conquest service about 4 or 5 times.

    Looks like a memory leak but after a few days digging logs I couldn't find any explanations or serious leads.


    Since yesterday evening, I put an older version of dgate64.exe (1.4.19 d2) and it seems to be stable now, no more RAM eating or abnormal CPU use.



    Question 1: I would like to know if some of you are using latest 1.5.0b with MS SQL and if you have some issues too, or if you set specific settings.


    I saw there are big additions in 1.5.0, especially multi threaded moves and forwards, don't know if this is related.


    Question 2: Other issue but I guess that is not a new one (because before I set the new server, I had one instance per modality, and I decided to move all to a single instance).


    Queries give no results if I check more than 1 modality as a filter (eg. DX + MR : 0 results ; no filter 25 results).


    I tried the procedure described in the following thread, without success: no change it works for Horos and Radiant but is not functionnal for MicroDicom, I can't figure out why this is different.

    OSIRIX/HOROS problem with multiple Modalities search


    Any help or feedback will be appreciated, clinicians are going to hate me for this change, maybe it is too late :)


    Thanks

  • Script library

    Hi


    OK, it was due to missing quotes for the new modality.


    Syntax is
    if Data.Modality~='CR' then Data.Modality = 'CR' end


    Tested the LUA importconverter as well -> perfect.


    Many thanks Marcel, and merry Christmas !

  • Script library

    Thanks Marcel.


    I added the lua line in dicom.ini but i haven't checked if it works yet.


    When i try to use the code in the lua box, if i hit the "test syntax" button an error occured (lua syntax error... unfinished line...). If i click "start modify", no error but no change in modality.


    Any other suggestion ?

  • Script library

    Hi Marcel,


    For some reason i have some Rx images that arrive to conquest with bad modality.
    I want to rewrite modality in every incoming dicom if modality is not CR.


    Do you think my import converter is OK ?


    Code
    ImportConverter1 = ifnotequal "%m", "CR"; set "%m", "CR";


    Besides, i want to change past errors and now it asks for a lua script...


    Could you show me how to change modality of a serie using lua box in the "browse database" tab ?


    Thanks,

  • Can't display, edit, delete image entry.

    Hi Marcel,


    That's kinda weird, i use the same database, SQlite.
    As i said before, i also tried with DBASE III on a test server and had the same issue.


    I can use the old GUI as a workaround, but i can't understand what happen...
    I was thinking to a user right problem but couldn't find anything that could explain.


    Could it be related to the OS version ? I can't remember but it is possible that the issue appeared after i migrated on my Windows 2012 server.


    I keep in touch if i can find any explanation.


    Thanks for your help Marcel.