Conquest DICOM server 1.4.19beta release

  • Hi Conquest users,


    After a long wait, here is the beta release 1.4.19beta. For simplicity this is a windows only release without sources. These are the major changes: 1) a complete recoding of the browser in the GUI for speed and lots of new options such as flexible DICOM modification, server speedup, JpegLS and better Jpeg2000
    compression (thanks Bruce Barton), a completely scripted web interface with a new interactive viewer, and options for a new flexible anonymisation server.


    You can download it here:


    http://ingenium.home.xs4all.nl…r/dicomserver1419beta.zip


    Almost all reported issues on windows have been addressed. Linux and source users will have to wait a while - repackaging and documenting is a daunting task.


    Can you please try it? Reporting of issues will be greatly appreciated.


    Marcel

  • Here bug reports for 1.4.19beta are collected.


    Stability report: Excellent so far except rare jpeg2000 and jpegls crashes on atypical images.


    These are leftover bugs. Earlier reported Linux bugs are still listed as only a windows release has been made.
    1) make same distro compilable for Linux; in 1.4.19beta2
    2) odbc config option of GUI is broken under windows 7; server runs (slow) with sqlexpress 2008, Works in 32 bits when runs as administrator. <Will not fix>
    3) Modify the maklinux_mysql file there should be -ldl in compile command (Andrey); in 1.4.19beta2
    4 ) Modify the conquest-pacs.sh. The restart line has hardcoded conquest user. So better if the line will be --chuild $PACSUSER instead of --child conquest (Andrey). In 1.4.19beta2
    5) I think the maklinux_mysql the commands cp dgate /usr/lib/cgi-bin should be changed to ln -s so everything will be installed at the same place and dgate and config for it (Andrey). In 1.4.19beta3
    6). Also I modified conquest-pacs.sh name changed it to nconquest-pacs.sh so it's start after mysql (Andrey). In 1.4.19beta2
    7) integrate weasis information from Richard Lawrence; updated weasis starters and info in 1.4.19beta3
    8 ) process Linux release by Andrey Murakhtaev. In 1.4.19beta2
    9) add LittleEndianExplicit to GUI connecting to dgate
    10) newweb doesn't like patientid with spaces in it. Fixed in 1.4.19beta3
    11) Anton Iarkin: Maklinux_mysql: At the end of g++ string to be the -ldl option; in 1.4.19beta2
    12 ) idem: Maklinux_postgresql: Link to libraries should be /usr/include/postgresql, but not /usr/local/pgsql/include and at the end of g++ string to be the -ldl option. (duplicate of 3); in 1.4.19beta2
    (I tested conquestlinux1417d in Ubuntu 14.04.) If it helps, I'll be happy. Thank you for "Conquest"!
    13) Improve safety of sqlite by regular flushing
    14) JpegLS crash on image where highest value exceeds BitsStored; partly fixed in 1.4.16beta3
    15) Check these changes: 20140608 mvh Added BYTE PCid to CStoreRSP::Write; This change is under review: pcid does not seem to get the right value


    newly found:
    16) jpegls not implemented/documented for WADO. In 1.4.19beta3.
    17) browser sorting fails on postgres (table goes blank). Fixed in 1.4.19beta3
    18 ) brainix sample data fails to load from zip; maybe because of French folder names
    19) sql server and access database ODBC install only work for 32 bits of dgate (e.g. disable 64 bits or create ODBC entry by hand)
    20) document install of newweb (copy newweb folder as is into xampp\cgi-bin); add auto install?
    21) install second server when service of first runs gives trouble - do an echo on the install button and fail if found
    22) crash of JpegLs and Jpeg2000 compression on Mirada state file with multiframe binary data. JpegLS fix in 1.4.19beta3
    23) last line of dicom.ini does not read if no CR (IndexDBF = 50)
    24) Linux compile: LoadAndDeleteDir(TempPath, NewPatid, PDU); should be LoadAndDeleteDir(TempPath, NewPatid, PDU, Thread); in 1.4.19beta2
    missing counters ImagesToBmpFromGui ImagesToJpgFromGui, used ImagesToGifFromGui needed in 3 places, Global, ServerTask and lua read in 1.4.19beta3 (Bruce)
    25) Jukebox number not displayed in browser. Fixed in 1.4.19beta3
    26) crash of JpegLS on some images with very bright isolated pixels
    27) syntax check button cannot be used in large dbaseiii servers as it starts a new server exe which takes too long.
    28 ) not all events (association/endassocation) and lua destroy, reject and retry are documented in manual. In 1.4.19beta3
    29) update overview.lua to properly exercise 1.4.19 commands
    30) Check that HAVE_USLEEP is enabled for sqlite; in 1.4.19beta2
    31) Publish about Docker release of Lance
    32) get_remotea in web interface (pm). In 1.4.19beta3
    33) write_sop bug ACRNEMA.MAP hardcoded (pm). In 1.4.19beta3
    34) Linux chmod to be documented. In 1.4.19beta3
    35) path to anoymize_script is incorrect in anonymizer/trial_collect.lua. In 1.4.19beta3
    36) See Joerg's post about umloat characters with fix in weasisseries/studyxml. In 1.4.19beta3
    37) newuids requires new database layout - needs URGENT FIX. In 1.4.19beta3
    38 ) Bruce needs to add delay in socket to speedup on OSX. In 1.4.19beta3
    39) Exportconverters broken. In 1.4.19beta3
    40) Change PatientID also changes PatientName; this should be reverted to PatientID only. In 1.4.19beta3
    41) gui freeze on window 10 - not reproduced so far
    42) cleanup layout of borders on install page. In 1.4.19beta3
    43) GUI fails to run as user after install as service on server2012; does run as admin.
    44) take browser database building generated messages out of server log. Fixed in 1.4.19beta3
    45) document which converters e.g. MoveDeviceConverter0 modify images and should not be used for logging. In 14.19beta3
    46) Add new film layouts to dicom printing 8x10, 10x12,10x14,11x14,11x17,14x14. Fixed in 1.4.19beta3
    47) Implement destroy in QueryResult Converters. In 1.4.19beta3


    Marcel

  • This is a place holder for guidance how to use new feautures.


    1) Updating: you can copy executables as usual to update, but select the right lua51.dll for 32/64 bits systems from the install32/64 folder.


    2) If you update and use zerobrane studio, leave only one of dgate.exe (32 bits) or dgate64.exe (64 bits) in the server folder.


    3) To install newweb as only web interface: copy the contents of webserver\cgi-bin\newweb into cgi-bin of your xampp install. That makes http://127.0.0.1/cgi-bin/dgate.exe the new interface. Do make sure the acrnema.map referred in cgi-bin/dicom.ini to has the correct entry for the server itself, e.g. "CONQUESTSRV1 127.0.0.1 5678 un"


    Marcel

  • Hi Marcel


    I am facing two errors while using the new beta release. The Database generation took quite a long time as compared to previous versions (Screenshot : Conquest Database Error)



    And secondly on Browse Database Tab i am also getting an error(Screenshot : Conquest Error).



    And thirdly i cannot change the AET of PACS server form the GUI.(screenshot : Conquest AET).




    --HM

  • Hi,


    Apparently dgate was running when you installed, but not the GUI. It that happens it reuses the same port and things go wrong, as stated in the manual.


    1) this is normal: database already exists - apparently you create two servers looking at the same database. Use SQLite to be safe to test.
    2) this happens when dgate is not running properly or (more likely in your case) you talk to a previous version of dgate. Set the port to a different number when you install
    3) this happens when a dgate runs that is not under control of the GUI. Set the port to a different number when you install


    regards,


    Marcel

  • I tested all database drivers and they all work fairly well. Performance tests will follow.


    Marcel


    INSTALL POSTGRESS
    postgres 9.5.1-1 for windows x64 62 MB
    installed with all defaults, password <pw>


    INSTALL XAMPP/MARIADB
    xampp-win32-5.6.19-0-VC11-installer 110 MB
    installed to c:xampp; includes 10.1.10-MariaDB
    run xampp control panel as admin
    make apache service
    make mysql service
    start xampp
    start mysql


    INSTALL SQLSERVER EXPRESS
    download sqlserver express SQL server 2014
    wants ms account X
    http://www.hanselman.com/blog/DownloadSQLServerExpress.aspx
    download takes 10 minutes.... 853 MB
    sql server
    start install
    disabled management tools
    named instance SQLExpress
    mixed mode
    SA password <pw>
    services - sqlbrowser - start (maybe needed because I disabled it at setup, needed at all if you guess the name?)


    FRESH INSTALL CONQUEST
    downloaded dicomserver1419beta.zip 15 MB
    md c:\dicomserver
    copy stuff from zip there


    INSTALL CONQUEST WITH MYSQL
    conquestdicomserver.exe
    select mysql
    save config
    make mysql database
    <conquest>
    <>
    test
    initialize database
    browser works with sorting


    INSTALL CONQUEST WITH POSTGRES
    conquestdicomserver.exe
    select postgres
    save config
    make postgress database
    <127.0.0.1>
    <conquest>
    <conquest>
    <geheim>
    test
    initialize database
    browser works but sorting fails, click again to unsort


    INSTALL CONQUEST WITH DBASEIII
    conquestdicomserver.exe
    select dbaseIII
    save config
    initialize database
    browser works but sorting fails without warning


    INSTALL CONQUEST WITH SQLITE
    conquestdicomserver.exe
    select Sqlite
    save config
    initialize database
    browser works


    INSTALL CONQUEST WITH SQLSERVER
    need to switch to 32 bits mode
    delete dgate64.exe and install64/dgate64.exe
    conquestdicomserver.exe
    select SQL server
    save config
    make ODBC datasource
    servername: <MACHINENAME\SQLEXPRESS>
    db name: <conquest>
    login: <conquest>
    pw: <conquest1415> (default generated password)
    SA pw: <pw>
    test
    initialize database
    browser works


    INSTALL CONQUEST WITH ACCESS DATABASE
    need to switch to 32 bits mode
    delete dgate64.exe and install64/dgate64.exe
    conquestdicomserver.exe
    select Access database
    save config
    make ODBC datasource
    test
    initialize database
    browser works

  • Here is the tested database performance, same tests as described in the manual, copying 4.4 million image records without the actual images.


    Marcel


    SQLSERVER (32 bits dgate.exe) - fast but resource hungry
    CPU activity during copying - about 50%; Memory use - 1.6 - 2 GB for sql server;
    Copy time 1h57m
    Database size 3.13 GB
    Query speed vs #records: 28ms + 0.084 ms/record


    MARIADB - relatively slow writing but extremely fast reading - without optimizing
    CPU activity 15%; Memory use very small
    Copy time 8h47 - Note this MySQL default setting; can be optimied
    Database size 3.7 GB
    Query speed: 8 ms + 0.067 ms / record


    POSTGRES - very fast writing, some read overhead for short queries
    CPU activity 40% or less; Memory use very small
    Copy time 1h57
    database size: 2.87 GB
    Query speed: 77ms + 0.067 ms / record


    DBASEIII (Using IndexDBF = 50 in dicom.ini) - still quite speedy
    CPU activity 40%; Memory use 100 Mb
    Copy time 3h15
    Database size 6.09 GB
    Query speed: 12ms + 0.1 ms / record


    SQLITE v 3.11.1 (built-in) - relatively slow
    CPU activity - about 42%; Quite some offline activity (inbetween associations)
    Memory use max 150 MB
    Copy time 6h47m
    Database size 2.9 GB
    Query speed vs #records: 68ms + 0.33 ms/record


    [b]NULL DRIVER (32 bits dgate.exe)[/u] - just a baseline test
    CPU activity - about 50%
    Copy time 1h8m
    No database is created.

  • Quote from marcelvanherk

    This is a place holder for feature requests.


    Hello Marcel,
    thank you again for this perfect and wonderful chef d'oevre of dicom pacs. It works like a swiss watch for more than 10 years for me.
    May I again ask you a favor? From time to time I have to aggregate two or more pacs into one.
    Would it be possible to integrate a destructive move, which really deletes the moved dicom files in the source? And leaves the rest in the source or moves it into a folder "not readable"?
    If I have only one file inside 1 million files, it takes a very long tome to filter out exactly this one corrupt file.
    Thank you again and happy easter!


    Georg

  • Hi,


    the incoming folder is destructive, so by pointing the watchfolder from one onto the others data that may work, although it may choke on so much data. I guess a lua script may be the solution where you string dicommove and dicomdelete commands. The question is if you can detect erroneous files this way. So maybe this could work (pseudo code):


    Edit: the lua Read command returns 1 on success.


    list patients
    list files per patient
    read file
    test it somehow
    if ok
    script('forward to AE')
    dicomdelete
    else
    log issue
    end


    regards,


    Marcel

  • Dear Marcel,


    I have used the conquest suite on many different machines for over 10 years now, very successfully, and I cannot thank You enough for all the fantastic work.
    As I got more familiar with conquest I have extended my demands on it in parallel, and now ran into a few issues for which I hope you will be able to recommend a good strategy:


    I have now 400 persons in our research system with 3000-1000 images from 1-3 studies and 7-12 series each; this is expected to increase to 4-5000 persons once we go multicenter.
    1. The series needed for research are automatically retrieved using research database-generated batch files from our hospital PACS (or in the future from our global repository) once a patient has consented to using the series.
    2. In a second step I am using batch files which move the data to a pseudonymization instance, where they are modified using the anonymize_script.lua generating newuids.
    3. In a third step series are sent to the instance in our research system where series are stored and backed up and converted to nifti formatted files for later use in whole brain analyses tools. One batch file handles one command on one series.


    So I have PACS -> CQ1_CLEARNAME -> CQ2_PSEUDO -> CQ3_RESEARCH (CQ1 and CQ2 reside on the same Windows 7 Pro 64 bit machine with 32GB RAM CPU E5 with SSD as System Drive and 4 TB external USB3 HD).


    Everything went well moving the series from CQ1 to CQ2. The Pseudonymization step went well for the initial 1 mio or so then the SQLLITE started throwing errors of not being able to delete images from the database. Rerunning my batch scripts picked up remaining images but allocated different newuids to studies and series so that I now have a huge amount of split studies and series.


    I found the reason why SQLLITE threw the errors, even on a SSD drive above 1 GB SQLLITE in multithread simply was not fast enough to modify the entries because the entries were still locked by the previous step. I have since moved CQ2_PSEUDO to MYSQL with slightly less initial performance which picked up significantly as it went into > 1 mio entries and nicely, so far no errors.


    Now to reduce error sources I would like to change the anonymize_script.lua to a pseudonymize_script.lua which can lookup up patient_id and patient_pseudonym in a list file and still maintain de_pseudonymization capability so any series that is sent to CQ2_PSEUDO using --moveseries/--movestudy/--movepatient gets automatically pseudomyzed correctly by an import filter but can also be retrieved back.


    I just cannot get my head around how to change the scripting to do this in 1.4.19b.


    If you find the time I appreciate Your help,


    Regards


    Julian

  • Hi Julian,


    with which version of conquest did you have the SqlLite issue? The mechanisms for pseudomyzation are in the new lua/anonymizer system that is released with 1.4.19beta. In short, the use provides a (one or more) CSV lists of patient ID vs trial patient ID. The trial patients ID is stored on the UID remapping table (just like all UIDs) so you can go back at any time or add data without causing split studies as long as the database is kept.


    I am presenting about this stuff at ICCR2016 in London in June. I will post the full abstract after that.


    regards,


    Marcel

  • Hi Marcel,


    the SQLLITE errors occurred with 1.4.19b (see below). I am worried because my batch files definitely do not try to assign the same subject to two different pseudonyms. The database lock problem has been described years ago, with a 1 s wait for each lock error quickly building up to the 5 s time out of SQLLITE, so I don't know if this applies here.


    I am regenerating MAG0 in a new instance using MYSQL INNDB with my.ini set to "huge" and 4 GB RAM allocation and will run my modify batch scripts. Actually there are 3200 batch scripts, for each series separately -> maybe that is a mistake ? Are the new study ids being used for the sereis even if I choose reversible =false ?
    Example:
    f:\dicom\1.4.19b\dgate64.exe "--modifyseries:0012345678,1.3.12.2.1107.5.2.12.21228.30000014081305203421800007465,lua/anonymize_script.lua(zz00)"


    Regards



    Julian


    --- PacsTrouble.log ---
    ...
    20160401 05:01:18 ***Refused to enter inconsistent link PatientID into DICOMStudies: PatientID = 'am02' StudyInsta = '1.2.826.0.1.3680043.2.135.736046.54881188.7.1459461006.87.1', Old='ed63', Refused='am02'
    20160401 05:01:18 ***Error saving to SQL: am02\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459466241.60.77_0004_000335_1459479678ea28.dcm
    20160401 05:01:18 ***Error entering object into server: f:\dicom\1.4.19b\data\0010492141\1.3.12.2.1107.5.2.12.21228.30000014012406284323400008010_0004_000335_14591690118c94.dcm
    20160401 05:01:40 ***SQLITEExec error: database is locked
    20160401 05:01:40 ***Error saving to SQL: ed63\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459466241.60.77_0004_000328_1459479633ea13.dcm
    20160401 05:01:40 ***Error entering object into server: f:\dicom\1.4.19b\data\0010492141\1.3.12.2.1107.5.2.12.21228.30000014012406284323400008010_0004_000328_14591690208fab.dcm
    20160401 05:01:45 ***SQLITEExec error: database is locked
    20160401 05:01:50 ***SQLITEExec error: database is locked
    20160401 05:01:50 ***SQLITEExec error: database is locked
    ...


    --- serverstatus.log ---
    [P140904B] 20160401 05:01:15 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:15 Deleting database entry for image: f:\dicom\1.4.19b\data\0010492141\1.3.12.2.1107.5.2.12.21228.30000014012413450140600000097_0006_000742_145916899286e0.dcm
    [P140904B] 20160401 05:01:15 Modified image: f:\dicom\1.4.19b\data\na26\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459451441.182.19_0004_000544_1459479675ea25.dcm
    [P140904B] 20160401 05:01:15 Deleting file: f:\dicom\1.4.19b\data\0010478126\1.3.12.2.1107.5.2.12.21228.30000015012906232282800008373_0004_000544_14591688165bdb.dcm
    [P140904B] 20160401 05:01:15 ***SQLITEExec error: database is locked
    [P140904B] 20160401 05:01:15 ***Error saving to SQL: am02\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459466241.60.77_0004_000333_1459479606ea0d.dcm
    [P140904B] 20160401 05:01:15 ***Error entering object into server: f:\dicom\1.4.19b\data\0010492141\1.3.12.2.1107.5.2.12.21228.30000014012406284323400008010_0004_000333_14591690118c73.dcm
    [P140904B] 20160401 05:01:16 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:16 Deleting database entry for image: f:\dicom\1.4.19b\data\0010478126\1.3.12.2.1107.5.2.12.21228.30000015012906232282800008373_0004_000545_14591688125af3.dcm
    [P140904B] 20160401 05:01:17 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:17 Deleting database entry for image: f:\dicom\1.4.19b\data\0010492141\1.3.12.2.1107.5.2.12.21228.30000014012406284323400008010_0004_000334_1459169021900f.dcm
    [P140904B] 20160401 05:01:17 ***Refused to enter inconsistent link PatientID into DICOMStudies: PatientID = 'am02' StudyInsta = '1.2.826.0.1.3680043.2.135.736046.54881188.7.1459461006.87.1', Old='ed63', Refused='am02'
    [P140904B] 20160401 05:01:17 ***Error saving to SQL: am02\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459466241.60.77_0004_000334_1459479677ea26.dcm
    [P140904B] 20160401 05:01:17 ***Error entering object into server: f:\dicom\1.4.19b\data\0010492141\1.3.12.2.1107.5.2.12.21228.30000014012406284323400008010_0004_000334_1459169021900f.dcm
    [P140904B] 20160401 05:01:18 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:18 Deleting database entry for image: f:\dicom\1.4.19b\data\0010492141\1.3.12.2.1107.5.2.12.21228.30000014012406284323400008010_0004_000335_14591690118c94.dcm
    [P140904B] 20160401 05:01:18 ***Refused to enter inconsistent link PatientID into DICOMStudies: PatientID = 'am02' StudyInsta = '1.2.826.0.1.3680043.2.135.736046.54881188.7.1459461006.87.1', Old='ed63', Refused='am02'
    [P140904B] 20160401 05:01:18 ***Error saving to SQL: am02\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459466241.60.77_0004_000335_1459479678ea28.dcm
    [P140904B] 20160401 05:01:18 ***Error entering object into server: f:\dicom\1.4.19b\data\0010492141\1.3.12.2.1107.5.2.12.21228.30000014012406284323400008010_0004_000335_14591690118c94.dcm
    [P140904B] 20160401 05:01:18 Modified image: f:\dicom\1.4.19b\data\vg11\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459446889.948.13_0005_000900_1459479669ea21.dcm
    [P140904B] 20160401 05:01:18 Deleting file: f:\dicom\1.4.19b\data\0010472483\1.3.12.2.1107.5.2.12.21228.30000014052105160614000009671_0005_000900_145916879255e3.dcm
    [P140904B] 20160401 05:01:20 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:20 Deleting database entry for image: f:\dicom\1.4.19b\data\0010492141\1.3.12.2.1107.5.2.12.21228.30000014012406284323400008010_0004_000336_14591690219030.dcm
    [P140904B] 20160401 05:01:21 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:21 Deleting database entry for image: f:\dicom\1.4.19b\data\0010472483\1.3.12.2.1107.5.2.12.21228.30000014052105160614000009671_0005_000901_145916878452c7.dcm
    [P140904B] 20160401 05:01:21 Modified image: f:\dicom\1.4.19b\data\vg11\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459446889.948.13_0005_000901_1459479681ea29.dcm
    [P140904B] 20160401 05:01:21 Deleting file: f:\dicom\1.4.19b\data\0010472483\1.3.12.2.1107.5.2.12.21228.30000014052105160614000009671_0005_000901_145916878452c7.dcm
    [P140904B] 20160401 05:01:22 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:22 Deleting database entry for image: f:\dicom\1.4.19b\data\0010472483\1.3.12.2.1107.5.2.12.21228.30000014052105160614000009671_0005_000902_14591687935601.dcm
    [P140904B] 20160401 05:01:22 Modified image: f:\dicom\1.4.19b\data\vg11\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459449035.413.9_0004_000611_1459479682ea2b.dcm
    [P140904B] 20160401 05:01:22 Deleting file: f:\dicom\1.4.19b\data\0010472483\1.3.12.2.1107.5.2.12.21228.30000013112106380340600006397_0004_000611_14591686953673.dcm
    [P140904B] 20160401 05:01:22 Modified image: f:\dicom\1.4.19b\data\lj84\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459472694.135.98_0011_000124_1459479682ea2c.dcm
    [P140904B] 20160401 05:01:22 Deleting file: f:\dicom\1.4.19b\data\0010503141\1.3.12.2.1107.5.2.12.21228.30000014030606231656200013432_0011_000124_1459169212b8e0.dcm
    [P140904B] 20160401 05:01:24 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:24 Deleting database entry for image: f:\dicom\1.4.19b\data\0010472483\1.3.12.2.1107.5.2.12.21228.30000013112106380340600006397_0004_000612_145916866632f8.dcm
    [P140904B] 20160401 05:01:24 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:24 Deleting database entry for image: f:\dicom\1.4.19b\data\0010503141\1.3.12.2.1107.5.2.12.21228.30000014030606231656200013432_0011_000125_1459169211b8b7.dcm
    [P140904B] 20160401 05:01:24 Modified image: f:\dicom\1.4.19b\data\mo54\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459479027.650.22_0008_000015_1459479677ea27.dcm
    [P140904B] 20160401 05:01:24 Deleting file: f:\dicom\1.4.19b\data\0010513860\1.3.12.2.1107.5.2.12.21228.30000013082705164823400005270_0008_000015_1459169252c303.dcm
    [P140904B] 20160401 05:01:25 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:25 Deleting database entry for image: f:\dicom\1.4.19b\data\0010513860\1.3.12.2.1107.5.2.12.21228.30000013082705164823400005270_0008_000016_1459169252c304.dcm
    [P140904B] 20160401 05:01:25 Modified image: f:\dicom\1.4.19b\data\mo54\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459479027.650.22_0008_000016_1459479685ea2e.dcm
    [P140904B] 20160401 05:01:25 Deleting file: f:\dicom\1.4.19b\data\0010513860\1.3.12.2.1107.5.2.12.21228.30000013082705164823400005270_0008_000016_1459169252c304.dcm
    [P140904B] 20160401 05:01:26 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:26 Deleting database entry for image: f:\dicom\1.4.19b\data\0010513860\1.3.12.2.1107.5.2.12.21228.30000013082705164823400005270_0008_000017_1459169252c305.dcm
    [P140904B] 20160401 05:01:27 Modified image: f:\dicom\1.4.19b\data\vg11\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459449035.413.9_0004_000612_1459479687ea31.dcm
    [P140904B] 20160401 05:01:27 Deleting file: f:\dicom\1.4.19b\data\0010472483\1.3.12.2.1107.5.2.12.21228.30000013112106380340600006397_0004_000612_145916866632f8.dcm
    [P140904B] 20160401 05:01:28 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:28 Deleting database entry for image: f:\dicom\1.4.19b\data\0010472483\1.3.12.2.1107.5.2.12.21228.30000013112106380340600006397_0004_000613_14591686963694.dcm
    [P140904B] 20160401 05:01:28 Modified image: f:\dicom\1.4.19b\data\vg11\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459449035.413.9_0004_000613_1459479688ea32.dcm
    [P140904B] 20160401 05:01:28 Deleting file: f:\dicom\1.4.19b\data\0010472483\1.3.12.2.1107.5.2.12.21228.30000013112106380340600006397_0004_000613_14591686963694.dcm
    [P140904B] 20160401 05:01:28 Modified image: f:\dicom\1.4.19b\data\mo54\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459477503.99.65_0006_000070_1459479673ea23.dcm
    [P140904B] 20160401 05:01:28 Deleting file: f:\dicom\1.4.19b\data\0010513860\1.3.12.2.1107.5.2.12.21228.30000013082708253295300000000_0006_000070_1459169258c44c.dcm
    [P140904B] 20160401 05:01:30 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:30 Deleting database entry for image: f:\dicom\1.4.19b\data\0010472483\1.3.12.2.1107.5.2.12.21228.30000013112106380340600006397_0004_000614_14591686673319.dcm
    [P140904B] 20160401 05:01:31 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:31 Deleting database entry for image: f:\dicom\1.4.19b\data\0010513860\1.3.12.2.1107.5.2.12.21228.30000013082708253295300000000_0006_000071_1459169258c44d.dcm
    [P140904B] 20160401 05:01:31 Modified image: f:\dicom\1.4.19b\data\lj84\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459472093.375.18_0005_000122_1459479691ea34.dcm
    [P140904B] 20160401 05:01:31 Deleting file: f:\dicom\1.4.19b\data\0010503141\1.3.12.2.1107.5.2.12.21228.30000014030606231656200009562_0005_000122_1459169109acc5.dcm
    [P140904B] 20160401 05:01:32 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:32 Deleting database entry for image: f:\dicom\1.4.19b\data\0010503141\1.3.12.2.1107.5.2.12.21228.30000014030606231656200009562_0005_000123_1459169109acc6.dcm
    [P140904B] 20160401 05:01:32 Modified image: f:\dicom\1.4.19b\data\lj84\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459472093.375.18_0005_000123_1459479692ea35.dcm
    [P140904B] 20160401 05:01:32 Deleting file: f:\dicom\1.4.19b\data\0010503141\1.3.12.2.1107.5.2.12.21228.30000014030606231656200009562_0005_000123_1459169109acc6.dcm
    [P140904B] 20160401 05:01:33 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:33 Deleting database entry for image: f:\dicom\1.4.19b\data\0010503141\1.3.12.2.1107.5.2.12.21228.30000014030606231656200009562_0005_000124_1459169109acc7.dcm
    [P140904B] 20160401 05:01:33 Modified image: f:\dicom\1.4.19b\data\lj84\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459472093.375.18_0005_000124_1459479693ea37.dcm
    [P140904B] 20160401 05:01:33 Deleting file: f:\dicom\1.4.19b\data\0010503141\1.3.12.2.1107.5.2.12.21228.30000014030606231656200009562_0005_000124_1459169109acc7.dcm
    [P140904B] 20160401 05:01:34 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:34 Deleting database entry for image: f:\dicom\1.4.19b\data\0010503141\1.3.12.2.1107.5.2.12.21228.30000014030606231656200009562_0005_000125_1459169109acc8.dcm
    [P140904B] 20160401 05:01:34 Modified image: f:\dicom\1.4.19b\data\lj84\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459472093.375.18_0005_000125_1459479694ea38.dcm
    [P140904B] 20160401 05:01:34 Deleting file: f:\dicom\1.4.19b\data\0010503141\1.3.12.2.1107.5.2.12.21228.30000014030606231656200009562_0005_000125_1459169109acc8.dcm
    [P140904B] 20160401 05:01:36 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:36 Deleting database entry for image: f:\dicom\1.4.19b\data\0010503141\1.3.12.2.1107.5.2.12.21228.30000014030606231656200009562_0005_000126_1459169109acc9.dcm
    [P140904B] 20160401 05:01:36 Modified image: f:\dicom\1.4.19b\data\lj84\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459472093.375.18_0005_000126_1459479696ea39.dcm
    [P140904B] 20160401 05:01:36 Deleting file: f:\dicom\1.4.19b\data\0010503141\1.3.12.2.1107.5.2.12.21228.30000014030606231656200009562_0005_000126_1459169109acc9.dcm
    [P140904B] 20160401 05:01:37 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:37 Deleting database entry for image: f:\dicom\1.4.19b\data\0010503141\1.3.12.2.1107.5.2.12.21228.30000014030606231656200009562_0005_000127_1459169109acca.dcm
    [P140904B] 20160401 05:01:37 Modified image: f:\dicom\1.4.19b\data\lj84\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459472093.375.18_0005_000127_1459479697ea3b.dcm
    [P140904B] 20160401 05:01:37 Deleting file: f:\dicom\1.4.19b\data\0010503141\1.3.12.2.1107.5.2.12.21228.30000014030606231656200009562_0005_000127_1459169109acca.dcm
    [P140904B] 20160401 05:01:38 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:38 Deleting database entry for image: f:\dicom\1.4.19b\data\0010503141\1.3.12.2.1107.5.2.12.21228.30000014030606231656200009562_0005_000128_1459169109accb.dcm
    [P140904B] 20160401 05:01:38 Modified image: f:\dicom\1.4.19b\data\lj84\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459472093.375.18_0005_000128_1459479698ea3c.dcm
    [P140904B] 20160401 05:01:38 Deleting file: f:\dicom\1.4.19b\data\0010503141\1.3.12.2.1107.5.2.12.21228.30000014030606231656200009562_0005_000128_1459169109accb.dcm
    [P140904B] 20160401 05:01:39 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:39 Deleting database entry for image: f:\dicom\1.4.19b\data\0010503141\1.3.12.2.1107.5.2.12.21228.30000014030606231656200009562_0005_000129_1459169109accc.dcm
    [P140904B] 20160401 05:01:39 Modified image: f:\dicom\1.4.19b\data\lj84\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459472093.375.18_0005_000129_1459479699ea3d.dcm
    [P140904B] 20160401 05:01:39 Deleting file: f:\dicom\1.4.19b\data\0010503141\1.3.12.2.1107.5.2.12.21228.30000014030606231656200009562_0005_000129_1459169109accc.dcm
    [P140904B] 20160401 05:01:40 ***SQLITEExec error: database is locked
    [P140904B] 20160401 05:01:40 ***Error saving to SQL: ed63\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459466241.60.77_0004_000328_1459479633ea13.dcm
    [P140904B] 20160401 05:01:40 ***Error entering object into server: f:\dicom\1.4.19b\data\0010492141\1.3.12.2.1107.5.2.12.21228.30000014012406284323400008010_0004_000328_14591690208fab.dcm
    [P140904B] 20160401 05:01:41 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:41 Deleting database entry for image: f:\dicom\1.4.19b\data\0010503141\1.3.12.2.1107.5.2.12.21228.30000014030606231656200009562_0005_000130_1459169109accd.dcm
    [P140904B] 20160401 05:01:41 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:41 Deleting database entry for image: f:\dicom\1.4.19b\data\0010492141\1.3.12.2.1107.5.2.12.21228.30000014012406284323400008010_0004_000329_14591690309347.dcm
    [P140904B] 20160401 05:01:41 Modified image: f:\dicom\1.4.19b\data\ed63\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459466241.60.77_0004_000329_1459479701ea3e.dcm
    [P140904B] 20160401 05:01:41 Deleting file: f:\dicom\1.4.19b\data\0010492141\1.3.12.2.1107.5.2.12.21228.30000014012406284323400008010_0004_000329_14591690309347.dcm
    [P140904B] 20160401 05:01:43 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:43 Deleting database entry for image: f:\dicom\1.4.19b\data\0010492141\1.3.12.2.1107.5.2.12.21228.30000014012406284323400008010_0004_000330_14591690208fcc.dcm
    [P140904B] 20160401 05:01:43 Modified image: f:\dicom\1.4.19b\data\wh57\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459460394.828.80_0006_000399_1459479703ea40.dcm
    [P140904B] 20160401 05:01:43 Deleting file: f:\dicom\1.4.19b\data\0010482848\1.3.12.2.1107.5.2.12.21228.30000010072205231315600007287_0006_000399_14591689327457.dcm
    [P140904B] 20160401 05:01:44 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:44 Deleting database entry for image: f:\dicom\1.4.19b\data\0010482848\1.3.12.2.1107.5.2.12.21228.30000010072205231315600007287_0006_000400_145916892270dc.dcm
    [P140904B] 20160401 05:01:44 Modified image: f:\dicom\1.4.19b\data\wh57\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459460394.828.80_0006_000400_1459479704ea41.dcm
    [P140904B] 20160401 05:01:44 Deleting file: f:\dicom\1.4.19b\data\0010482848\1.3.12.2.1107.5.2.12.21228.30000010072205231315600007287_0006_000400_145916892270dc.dcm
    [P140904B] 20160401 05:01:44 Modified image: f:\dicom\1.4.19b\data\na26\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459451441.182.19_0004_000545_1459479704ea42.dcm
    [P140904B] 20160401 05:01:44 Deleting file: f:\dicom\1.4.19b\data\0010478126\1.3.12.2.1107.5.2.12.21228.30000015012906232282800008373_0004_000545_14591688125af3.dcm
    [P140904B] 20160401 05:01:45 ***SQLITEExec error: database is locked
    [P140904B] 20160401 05:01:46 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:46 Deleting database entry for image: f:\dicom\1.4.19b\data\0010482848\1.3.12.2.1107.5.2.12.21228.30000010072205231315600007287_0006_000401_14591689327478.dcm
    [P140904B] 20160401 05:01:46 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:46 Deleting database entry for image: f:\dicom\1.4.19b\data\0010478126\1.3.12.2.1107.5.2.12.21228.30000015012906232282800008373_0004_000546_14591688316016.dcm
    [P140904B] 20160401 05:01:46 Modified image: f:\dicom\1.4.19b\data\ck90\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459436750.991.74_0006_001156_1459479692ea36.dcm
    [P140904B] 20160401 05:01:46 Deleting file: f:\dicom\1.4.19b\data\0010450667\1.3.12.2.1107.5.2.12.21228.30000011042812013809300001928_0006_001156_145916838512c4.dcm
    [P140904B] 20160401 05:01:46 Modified image: f:\dicom\1.4.19b\data\na26\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459451441.182.19_0004_000546_1459479706ea44.dcm
    [P140904B] 20160401 05:01:46 Deleting file: f:\dicom\1.4.19b\data\0010478126\1.3.12.2.1107.5.2.12.21228.30000015012906232282800008373_0004_000546_14591688316016.dcm
    [P140904B] 20160401 05:01:46 Modified image: f:\dicom\1.4.19b\data\lj84\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459472093.375.18_0005_000130_1459479706ea45.dcm
    [P140904B] 20160401 05:01:46 Deleting file: f:\dicom\1.4.19b\data\0010503141\1.3.12.2.1107.5.2.12.21228.30000014030606231656200009562_0005_000130_1459169109accd.dcm
    [P140904B] 20160401 05:01:49 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:49 Deleting database entry for image: f:\dicom\1.4.19b\data\0010450667\1.3.12.2.1107.5.2.12.21228.30000011042812013809300001928_0006_001157_14591683951660.dcm
    [P140904B] 20160401 05:01:49 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:49 Deleting database entry for image: f:\dicom\1.4.19b\data\0010478126\1.3.12.2.1107.5.2.12.21228.30000015012906232282800008373_0004_000547_14591688245e72.dcm
    [P140904B] 20160401 05:01:50 Importconverter-1.0 executes: newuids
    [P140904B] 20160401 05:01:50 Deleting database entry for image: f:\dicom\1.4.19b\data\0010503141\1.3.12.2.1107.5.2.12.21228.30000014030606231656200009562_0005_000131_1459169109acce.dcm
    [P140904B] 20160401 05:01:50 Modified image: f:\dicom\1.4.19b\data\ed63\1.2.826.0.1.3680043.2.135.736046.54881188.7.1459466241.60.77_0004_000330_1459479703ea3f.dcm
    [P140904B] 20160401 05:01:50 Deleting file: f:\dicom\1.4.19b\data\0010492141\1.3.12.2.1107.5.2.12.21228.30000014012406284323400008010_0004_000330_14591690208fcc.dcm
    [P140904B] 20160401 05:01:50 ***SQLITEExec error: database is locked
    [P140904B] 20160401 05:01:50 ***SQLITEExec error: database is locked

  • Hi,


    it is 1.4.19beta I assume. Can you point me to the database lock problem? I used the latest version of SqLite in the new release.


    The batch files look OK. I assume you run one at a time, not multiple in parallel.


    The lua script works as follows:
    lookup old patient ID in uidmods, if found: use newpatient ID there. If not store patientID change in UIDmods
    then do the same for all UIDs including study and seriesinstanceUID.


    So basically it does not matter if you use modifyseries or modifystudy, all images are processed the same. But, you cannot SPLIT an existing patient over mutiple patientIDs if reversible is set to true. And you can never split an existing studies over multiple studies.


    regards,


    Marcel

  • Hi Marcel,


    I found this old thread http://beets.io/blog/sqlite-nightmare.html which may be related.


    Yes, it is 1.4.19b. Also, I noted that by corporate enforcement of SOPHOS antivirus suite, ALL file access is screened by the online scanner, severely slowing database operations.


    Now running it with MYSQL it is slow but so far no errors. I noted in the serverstatus.log that there seem to be up to 4 images processed simultaneously.


    I do not run multiple batches in parallel, no.


    Thanks so far


    Julian

  • Hi Marcel,


    I am not sure my post got through, I was interrupted.


    Yes it is 1.4.19b. The SQLLITE lock time out problem has been described here http://beets.io/blog/sqlite-nightmare.html


    So far, with MYSQL everythings seems to work fine, from the logs apparentely up to 4 images are processed simultaneously.


    Also, our SOPHOS virus scanner is set to scan all file access (corporate policy) so it is slow.


    Thank You in advance


    Julian

  • Hi Julian,


    I indeed may not have HAVE_USLEEP enabled in my sqlite compile. Thanks for the report. Added it to the buglist.


    I wonder why four images are processed at the time - modifyseries is definitively doing images one by one.


    regards,


    Marcel

  • Hi Marcel,


    using MYSQL I had no more lock errors, which is great.


    PACS -> CQ1 (clear name) <-> CQ2 (pseudonym) -> CQ3 (research repository) works now.


    Now I only have a last task which may be possible using 1.4.19b:


    PACS -> CQ1 (clear name) <-> CQ2 (pseudonym) -> CQ4 (export)


    I have added a forth instance CQ4 in the research system which should only save series as pseudonym_seriesdesc.zip in a project folder and not keep local dicoms. Can CQ4 have different names/aliases which control the target project folder the series are exported to ?


    Background: I want to be able to move particular sets of series from CQ2 (pseudonym) to a project specific export folder using batch operations on CQ2 while CQ4 is on a machine with also provides a chroot environment for project analysts.


    Thanks again for your great work,


    Julian

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!