strange things with j2 compression

    Hi.


    Few days ago I was forced to introduce compression of images due to dwindling free space on our HDDs.
    So far I ran into 2 issues and after 2 days of debugging attempts I just ran out of ideas and decided to ask smarter people for help... hence this post.
    ____________________
    Lets start with some environment details:


    - 5 diagnostic machines ( 2x MRI, 2x CT, 1x USG), all sending images to 3 dgates (on 3 different ports) on one server.
    - soft version 1.4.17c
    - all incoming images are compressed using j2 ( dicom.ini: "IncomingCompression = j2")
    _____________________


    Issue nr 1:


    MOST of studies an just fine... but sadly few of them are not :( and by "few" I mean 3 out of 100ish.
    In 1st study there are 2 images (different series) which look like there was some problem in the middle of compression.. upper part of image is fine but lower is garbage.
    In 2nd study there are 4 images (same series) of which 2 look like 'compression went wrong' and 2 look like they do not belong to this study.
    In 3rd study there is 1 image that just does not belong there. This study was of a HEAD.. and 1 of images shows CHEST.
    How an image from completely different study landed inside another study is a mystery I just cant solve.


    Those 'alien' images have all tags of the study they landed in (I mean patient name/machine name, date etc.. all).
    Resending the study to the server fixes the issue, so the study itself is OK.
    No log entries indicate any problems. I looked everywhere.. dicom logs, system logs, database logs. Everything looks fine.


    Code
    20131206 09:18:47 Written file: f:\Data\50082507638\1.3.12.2.1107.5.1.4.54058.30000013120907061896800003599_0008_000043_1386577127073c.dcm

    20131206 09:18:47 [recompress]: recompressed with mode = j2 (strip=0)


    And that's it... no errors.


    Please help. I have absolutely no idea how this is happening. I suspect it might be related to the amount of incoming images at the same time... but the amount of devices sending them did NOT change. All I did was turning on the j2 compression.


    ____________________


    Issue nr 2 (unrelated to nr 1, or at least I think so)


    I was experimenting with how the dicom viewer ( Merge EFILM in this case) will handle compressed images.
    Images are already compressed on the server (with j2).
    When I use the 'un' for outgoing images everything is fine, so decompressing on the fly works.
    When I use 'j2' for outgoing images (by setting it in a arcnema.map) all I get on EFILM is garbage. Every single image looks like random black and white pixels.
    At first I thought it's a bug in the viewer but then I consulted their support team and they told me something like this:


    Quote

    There was a problem with transfer syntax UID - it was set to IMPLICIT_LE, but the incoming pixels were compressed. We edited it manually to .70 and then the study was displayed just fine. To put it simply - it looks like the server told us that images will be uncompressed but sent us compressed ones (with wrong tag)


    ____________________


    For me 2nd one is minor.. It's just a slight performance issue on server side (decompressing on the fly).. but the first one just kills me :(

    Hi,


    Focusing on Issue 1 first. This looks like the servers are mixing up pixel data from the compressor. The temporary files are stored as:


    tempDir\GenUID.dcm.


    Where GenUID is a new UID generated. If the 3 servers share the same tempDir (e.g., data\printer_files) AND the UIDPrefix is set to same value for all 3 servers this may happen sporadically! Hopefully setting the UIDPrefix to different values will solve this.


    Marcel

    For Issue 2.


    Are the misbehaving images recent and stored with "IncomingCompression = j2"? If you can look into the header on the server and see if the TransferSyntax is indeed set wrong it would appreciate.


    Marcel

    First of all - THANK YOU.. big time.


    1st issue - it looks like this is the case. If by "tempDir" you mean the system %temp% folder then yes - all 3 share the same one... and the UIDPrefix is indeed the same in all 3.
    I will change UIDPrefix then test it and let you know if that helped.



    2nd issue
    I was testing this on a study which was about 1y old. I compressed it few days ago by resending images with "IncomingCompression = j2" set in dicom.ini. (1.4.17c)


    As for the tags:
    (0002,0010) UI # 20 [1.2.840.10008.1.2.1_]


    In fact ALL images that were compressed have this tag (I guess it should end with .70 instead ?)
    It looks like they are compressed (about 50% of original size) but the tags remains unchanged. Tag is exactly the same before and after compression.

    Hi,


    Glad to be of help!


    I will look in the code for the second issue later. This looks like a proper bug. What happens when you send out something uncompressed with j2? Or j2 data with e.g. j3?


    Marcel

    Hi,


    I'm back with some test results.


    I used one of recent studies, so it was already compressed with j2 (but with wrong tag).


    1) j2 -> un
    I changed .ini to "IncomingCompression = un" and resend it to myself (I mean directly from server to server by loopback interface) using "dgate --movestudy" command
    As a result I got 2 times bigger files, as expected, and the tag remained the same.


    2) un -> j3
    Then after it was decompressed I changed incomingcompression to j3 and sent it again. And again I got about half sized images but the tag remained the same (1.2.840.10008.1.2.1_)

    Hi again.


    2 more tests:


    In both tests images used are j2 compressed (but with wrong tag)


    1) ( j2 -> j2)
    sending from server to Efilm using j2 setting in arcnema


    2) (j2 -> j3)
    sending from server to Efilm using j3 setting in arcnema


    In both cases all I got was garbage (black & white dots).
    In both cases tags were set to 1.2.840.10008.1.2
    Also in both cases serverstatus.log shows that accepted compression was "ui"

    Code
    20131212 14:20:53 Accepted compression: ui

    20131212 14:20:53 Sending file : f:\Data\.....


    What that even means? I don't see 'ui' listed as possible types in the manual :(


    --------------


    For issue 1 - I changed UIDPrefixes to be unique for all servers and so far (after 2 days) it looks like the problem disappeared... so thanks again :)

    Sure thing. All 3 files included:
    (sorry I didnt upload them as attachments but forum gives me this error: "Could not upload attachment to ./files/51731_7b574bf1bd2fe3e21ab7202ce77a10b3.")

    Code
    PACZK			127.0.0.1	2020	j2efilm4			192.1.1.52	104	    j2efilm2			192.1.1.54	4006	j2


    Code
    # This file contains configuration information for the DICOM server# Do not edit unless you know what you are doing[sscscp]MicroPACS                = sscscpEdition                  = Personal# Network configuration: server name and TCP/IP port#MyACRNema                = PACZKTCPPort                  = 2020# Reference to other files: known dicom servers; database layout; sopsACRNemaMap               = acrnema.mapkFactorFile              = dicom.sqlSOPClassList             = dgatesop.lst# Host(ignored), name, username and password for ODBC data sourceSQLHost                  = localhostSQLServer                =mpacs1Username                 =mpacs1Password                 =mpacs1Postgres                 = 1BrowseThroughDBF         = 1DoubleBackSlashToDB      =1UseEscapeStringConstants = 1# Configure databaseTruncateFieldNames       = 10MaxFieldLength           = 254MaxFileNameLength        = 255FixPhilips               = 0FixKodak                 = 0KeepAlive                = 0LargeFileSizeKB          = 4096PrintSquareLandscape     = 0UseKpacsDecompression    = 1ZipTime                  = 05:UIDPrefix                =1.2.826.0.1.3680043.2.1326.9EnableReadAheadThread    = 1PatientQuerySortOrder    = StudyQuerySortOrder      = SeriesQuerySortOrder     = ImageQuerySortOrder      = EnableComputedFields     = 1IndexDBF                 = 1PackDBF                  = 0LongQueryDBF             = 1000TCPIPTimeOut             = 300FailHoldOff              = 60RetryDelay               = 100RetryForwardFailed       = 0ImportExportDragAndDrop  = 1QueueSize                = 128WorkListMode             = 0WorkListReturnsISO_IR_100 = 1DebugLevel               = 0Prefetcher               = 0LRUSort                  = AllowTruncate            = DecompressNon16BitsJpeg  = 1UseBuiltInJPEG           = 1LossyQuality             = 95IgnoreOutOfMemoryErrors  = 0NoDICOMCheck             = 0PadAEWithZeros           = 0FileNameSyntax           =4# Configuration of compression for incoming images and archivalDroppedFileCompression   = j2IncomingCompression      = j2ArchiveCompression       = j2# Names of the database tablesPatientTableName         = DICOMPatientsStudyTableName           = DICOMStudiesSeriesTableName          = DICOMSeriesImageTableName           = DICOMImagesDMarkTableName           = DICOMAccessUpdatesRegisteredMOPDeviceTable = RegisteredMOPIDsUIDToMOPIDTable          = UIDToMOPIDUIDToCDRIDTable          = UIDToCDRID# Banner and host for debug informationPACSName                 =ARPACS1OperatorConsole          = 127.0.0.1# Configure email of error messagesMailHost                 = MailPort                 = smtpMailSignon               = MailFromName             = MailRcptName1            = MailCollectTime          = 1MailWaitTime             = 10# Configuration of disk(s) to store imagesMAGDeviceThreshhold      = 0MAGDeviceFullThreshHold  = 30IgnoreMAGDeviceThreshold = 0MAGDevices               = 1MAGDevice0               = f:\Data\NightlyCleanThreshhold   = 0


    oh, I just forgot to change it back. My initial tests were conducted with this option UNcommented.
    Then I thought I might try to comment it out in an attempt to force different type of syntax but it didn't work... result was the same in both cases.


    So far it looks like no matter what settings are set and no matter who is the recipient (conquest->conquest or conquest->efilm) the resulting images are always 1.2.840.10008.1.2


    It there a chance it's a bug or am I doing sth wrong here..? :(

    Sorry for delay.. beginning of new year means a TON of work for me :(


    I'm not sure if that's what you want but I set DebugLevel to "4" in dicom.ini and then tried to retrieve images from Conquest using Efilm. Here is what I got in .log


    "Host 'efilm' will not accept image" - this was probably some "SR" and thats why efilm rejected it.

    Hi


    I tried to launch "dgate --debuglevel:4" from command line but it closes almost immediately (1-2sec) with no error.
    Then I tried to launch it as a service using:

    Quote

    sc create debugtest binPath= "c:/test/DgateServ.exe /process dgate.exe --debuglevel:4"


    but it also almost immediately stopped working. In system log I see it stopped with error:

    Quote

    The debugtest service terminated with the following error:
    Access is denied.


    I honestly have no idea what it is about.. access TO WHAT? All file permissions looks fine.


    Isn't it the same as setting DebugLevel to "4" in dicom.ini ? (which also didn't work :( )

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!

Register Yourself Login