Conquest 1.4.16 beta released

Today, I released conquest 1.4.16beta. It can be found here:

http://www.xs4all.nl/~ingenium/dicom.html

Thanks again for all your support, and especially Bruce Barton for the JPEG and JPEG2000 code and lots of other changes. Note that much of the code has been touched. Please report any bugs ASAP.

And, oh I forgot to provide links to the previous 1.4.15 release as download. I will add them later

Marcel

These are the known bugs in 1.4.16beta:

1) Rare random high CPU load hang of a single server thread (bug recently reported on 1.4.14 and 1.4.15alpha, not on 1.4.13). Not recently confirmed. Consider fixed in 1.4.16

2) Seems like a warning will drop the connection (logs to be provided): viewtopic.php?f=33&t=1838. Not confirmed.

3) Default ports 5678 and 5679 seem in use by Vista; also port 1155. Modify port ranges and test availability of port on fresh install. Not changed yet.

4 ) Web interface fails to query studies and series in databases with really old definition (no patient name in study). Not fixed.

5 ) crash of virtualserver in other rare cases. still open. A virtual server move of over 1000 CT slices causes a timeout (or other error) on the c-move client, followed by a crash of the server. This problem occurs on a production server but not an test server...Consider fixed in 1.4.16

6) Printing does not work in 1.4.16beta. CQDICOM and dgate crash and takes the windows GUI along. Replacing the old cqdicom seems to fix the sending the problem, but the recieving dgate still crashes when the thread ends, after correctly writing the dicom object. Both issues may be syntaxalias related. Fixed in 1.4.16beta2 (thanks bcb)

7) Split series in two seems to make new studies instead of series. Fixed in 1.4.16beta4.

8 ) Double click in GUI query window populates the wrong edit box when there are [] in the data... Fixed in 1.4.16beta4.

9) Changing patient ID in an importconverter blocks the exportconverter. Fixed in 1.4.16beta4.

10) YBR_FULL and planar images problem in 1.4.16beta windows - solved in 1.4.16beta2.

11) If a virtualserver operation fails, the failed database records should be removed and an error message should be given - solved in 1.4.16beta3.

12) Accessionnumber should maybe not be a unique primary key in the worklist table. Fixed in 1.4.16

13) the <forward> Import converter may only be the first statement - solved in 1.4.16beta3.

14) the <call> Import converter did not pass called and calling - solved in 1.4.16beta3.

15) %Vxxxx,yyyy crashed server when item was longer than 255 chars - solved in 1.4.16beta3.

16) # and ; lines in a script file where still executed.- solved in 1.4.16beta3.

17) NKI multiframe data does not display in browser (extractframe(s) fails). - solved in 1.4.16beta3.

18) <Merge study> did not work; <delete study> etc causes endless retries. Fixed in 1.4.16beta4.

19) Compile issues for linux: http://forum.image-systems.biz/viewtopic.php?f=33&t=2240 (maklinux must be run with sudo).

20) Allow 0 group lenght as warning in trnsyn.cxx. Fixed in 1.4.16beta4.

21) frame control in GUI browser should start at 0 not one. Fixed in 1.4.16beta4.

22) make sqlite with doublebackslashtodb=1 impossible (may happen after a manual upgrade). Fixed in 1.4.16.

23) If a virtualserver hangs up (i.e., due to a forgotten VirtualServerPerSeries, and then asking for too many files), the connection with the client is not broken after the missing database entries have been removed form the database. Removal also tries to remove the files which is unnecessary. Fixed in 1.4.16beta5.

24) 1.4.16beta4: show incoming images crashes the server (not confirmed, not fixed).

25) Bigendian transfer does not seem to work for incoming c-store requests (command field already corrupted). To be tested.

26) ToJPG reverse blue and red (fixed in 1.4.16beta5); (all formats:) level and windowing overflows at high pixel values. Latter not fixed yet.

27) Update pscp to avoid hang. In 1.4.16beta5.

28) Default enable EnableComputedFields. Fixed in 1.4.16

29) add WADO server references to web interface dicom.ini. Fixed in 1.4.16

30) Typo in read of EnableInternalJPEG from windows GUI (save will now also enable it). Fixed in 1.4.16

31) DgateServ /start will give an exception when no access (windows 7) - should only be a message. Fixed in 1.4.16

32) Linux startup script changes of Mark Pearson. Seem to be ok in manual.

33) sqlite records could exceed given lenght: use strncpy to truncate them when reading them in NextRecord. Fixed in 1.4.16

34) FIXKODAK does not work consistently. Fixed in 1.4.16

35) There is a "nop" converter in the export options. Fixed in 1.4.16

36) The listed options for 7za in the web upload are incorrect. Fixed in 1.4.16

37) Enable delete options if image not there. Fixed in 1.4.16

38 ) put cleanup thread into dgate.exe if needed. Fixed in 1.4.16

39) restarting the source server during a clonedb action crashes the destination server. Fixed in 1.4.16

40) lock right new "only to logfile" button in 1.4.16releasecandidate. Fixed in 1.4.16

41) install as service and kill and restart server do not respect 'only log to file' button. Fixed in 1.4.16

42) web script variables from dicom.in only pass spaces when quoted e.g., variable = "sample 1". Future.

43) JPG and BMP output have their RGB swapped as BGR. Fixed in 1.4.16

44) deplane and depalette are not always called; depalette should be run first and kill planes. Fixed in 1.4.16

45) frame spinedit appears in GUI when #frames=1 (not when it is not set). Fixed in 1.4.16

46) jpeg data is not decompressed before transmittal. Fixed in 1.4.16

47) crash when cached query has zero items. Fixed in 1.4.16

48 ) 0x0020, x01209 not present in dgate.dic, and description of other computed items incorrect. Fixed in 1.4.16

49) queries with _ in item take very long: seen as wildcard. Fixed in 1.4.16

50) remove submissions table from release version. Fixed in 1.4.16

51) postgres sometimes randomly refuses a db.open under high load. Postgres errors only reported to debug log. Fixed in 1.4.16

52) default use BrowsethroughDBF for SQL server. Fixed in 1.4.16

Wish list:

1) A clean way of exiting, i.e., that counts active threads: http://forum.image-systems.biz…c.php?f=33&t=1843&start=0. Added dgate --safequit: in 1.4.16.

2 ) Allow underscores in AE: http://forum.image-systems.biz/viewtopic.php?f=33&t=1847. THIS WILL NOT BE ADDRESSED IN THE NEXT RELEASE. TO ENTER _, CLOSE SERVER, EDIT DICOM.INI, START SERVER.

3) Safer printing: thread-safe and no drops; http://forum.image-systems.biz/viewtopic.php?f=33&t=1844

4) A possible solution would be to add the current thread number to each line logged (not only to start and end of the thread) so that a simple sort before parsing the daily log files would overcome this problem.

5) Make LossyQuality configurable in ImportConverter, forward statement and/or queryconverter.

6) Add ArchiveConverter0. In 1.4.16.

7) Add MergeConverters. In 1.4.16beta2.

8 ) Display # objects in GUI browser. Fixed in 1.4.16beta4.

9) Add dgate --browsepatient: command that selects patient in windows GUI browser. In 1.4.16beta4.

10) <mkdir> <rm> converters. [,N] substring gives last N characters. In 1.4.16beta4.

11) <save frame %d to> <save frames %d, %d to> <save bmp/jpg/gif [frame %d][level %d][window %d][size %d] as file>. In 1.4.16beta4.

12) Add seriesdesc filter and script to <merge study> command. In 1.4.16beta4.

13) LZW compression (required for digital microscopy objects). Later

14) Add command to select/test data recieved today. Is in 1.4.16

15) Add commands to collect, anonymise and submit data. In 1.4.16beta4.

16) Add <format> clause to set IE converter. In 1.4.16beta4.

17) Allow multiple copies of images to be saved for test scripting <testmode>. Partial in 1.4.16beta4.

18) Allow disabling visual dicomserver GUI log (log directly to file). In 1.4.16.

19) Indicate when images are rewritten not added. In 1.4.16.

20) command line option to scan/delete a folder. In 1.4.16.

21) log file rotation when the server runs as a service. In 1.4.16.

22) Export menu options in conquest GUI. In 1.4.16.

23) self learning query cache prefetch; cache should use cached data of larger queries for smaller subqueries; cache local query data. Future.

24) use typecodes for our private tags (for explicit syntaxes) and put them in dict. dict fixed in 1.4.16

25) make ModalitiesInStudy a computed field. Future.

26) Long queries with computed fields will take forever. Should use 'count where'. Future. Use scripts to block there queries. But queries accelerated using (COUNT 1) in 1.4.16. Maybe also cache these items in the database: cleared when writing or deleting new image, set when querying with COUNT. This trick may also useful for ModalitiesInStudies.

27) Put cleanup thread in dgate.exe. In 1.4.16.

28 ) Allow setting some web variables like address and port from the scripted pages (could be done with frames).

29) "move series as AE from AE to AE" command. Later. "org" clause in forward commands. The later is fixed in 1.4.16

30) Forward studies older than xx days from MAGN to AE and move them to MAGM. Fixed in 1.4.16

31) echo command. Fixed in 1.4.16

32) test script option from GUI browser. Later

To get a list of bugs fixed in 1.4.16beta please have a look in the bug list for the 1.4.15 releases. Please report any bugs you find!

Hi,

I have re-made the previous 1.4.15 windows version downloadable. For this version, DICOM printing works.

Marcel

Here is a patch release which fixes printer support.

ftp://ftp-rt.nki.nl/outbox/Mar…erver/patch_1416beta2.zip.

It also adds a <format> clause to the <set> importconverter statements, e.g.,:

set x format "%%08d" to "%V0010,0020"

Marcel

Here is a patch release which fixes some issues:

ftp://ftp-rt.nki.nl/outbox/Mar…rver/1416beta3_update.zip

Marcel

Hi,

here is the next update (1.4.16beta4):

ftp://ftp-rt.nki.nl/outbox/Mar…rver_update_1416beta4.zip

It adds next buttons to the serverside viewer and adds quite a few new conversion options such as merge with script, sftp submission, and move with a script. These are yet undocumented.

See above for a complete list of fixes.

Marcel

Hi,

this is a bug: there is no anonymization script read yet, the newuids is a hardcoded 'sample' script. The "submit' study target ... password ... script ... IE converter can be passed a script, and should behave as instructed.

Marcel

Hi,

Here is the WIP - any comments are welcome. These otions are release in 1.4.16beta5 below.

I added WADO support and now have both a decent WADO server and simple client integrated. The WADO server uses virtualservers and may be used as a DICOM-WADO gateway for other DICOM systems. MPEG may be added most easily using the MJPEG-tools. The server does not yet anonimize (this is easy), crop (little less easy) and annotate (still less easy) the data.

I have added support for generating animated GIF's from DICOM objects. With help of Lennert.

I finished a web server upload of DICOM files (also zipped files). This uses new command dgate --addlocalfile:.

I added a directory monitor to allow automatic upload of DICOM and zip files. Together with the "submit" command this would allow efficient automatic sftp forwarding of large amounts of DICOM data between conquest servers.

The serverside viewer has some more buttons and knows if and how many frames there are. Todo: show #frames in visual image list for local and remote queries

Implemented caching of large identical queries to virtualservers at series or study level. WIll help us live with sloooooow kodak queries (about 200 records per s).

Implemented overlap of virtual server recieve and send operation with limit on memory use.

Marcel

Here is the beta5 update (manual is included):

ftp://ftp-rt.nki.nl/outbox/Mar…erver/Update1416beta5.zip

It contains source files (for linux) as well as binaries (for windows). The update should be merged with the 1.4.16beta release on the web page (link on top of this topic).

Updates: bug fixes, WADO server and client, converters: <merge>, <format> clause for <set>, <system>, <mkdir>, <rm>, <save frame>, <save bmp/gif/jpg>, <submit>, scripts in <forward>; dgate –browsepatient, --addlocalfile, --submit, --scheduletransfer; improved serversideviewer, caching of repetitive queries, enabled MAG0\incoming, upload from web server, optional overlap of get and send in virtualservers, animated GIF support.

Please report any bugs!

Marcel

Hi,

probably to use OS means to transfer the data and copy the database, and make sure that you are up to date before switching. The incoming folder will store and delete everything, but this will take more time for huge amounts.

Marcel

Thanks Bruce!

I will merge it in. I guess it is almost time for an official 1.4.16 release. I have just fixed windows7 issues.

Marcel

Sure,

as long as the number of changes is small, e.g., <20.

Marcel

Hi,

this is the release candidate of 1.4.16 for linux:

ftp://ftp-rt.nki.nl/outbox/Mar…16releasecandidate.tar.gz

Please provide feedback.

Marcel

Hi Bruce,

that is my issue with warning tracking - every new version shows more and forces you to change perfectly working code, with the probability of introducing bugs.

Here are my thoughts:

"Implicit conversion shortens 64-bit value into a 32-bit value" and most are from functions that return a size_t put into an int, shouldn't be a problem, but it could be.
Could be an issue with huge DICOM objects.

The second biggest warning is for "No previous prototype for 'some_function()'", easy to fix and unimportant.
I usually do not includes prototypes for loacl functions, I just make sure thay are above their application.

The next one, bothers me, is "Deprecated conversion from string constant to 'char*'", I would like to fix this one at least and there aren't that many of them (OK, I really would like to fix them all, but ...).
This is where strings are passed into fucntions call("jkjfkjdkds"). I have seen you changes these before.

The last one is 19 warnings generated by -Wglobal-constructors, 'some_global_function' requires global construction or destruction. I don't know much about this warning to know if I should care.
I have no clue.

Marcel

Hi,

can you send me the compiler output? I can then check them as well.

Marcel