Compression help!

Pacsdude · Sep 13th 2011

Current setup:
Conquest: v1.4.16
Windows Server 2008 x64
16GB of RAM
Two XEON 2GHz processors
Drives E & F for storage

I've got a situation where my dicom.ini file is set as this for compression:

Code

# Configuration of compression for incoming images and archivalDroppedFileCompression   = j2IncomingCompression      = j2ArchiveCompression       = as

The above settings works great for the majority of the studies that are stored on conquest for us because they arrive uncompressed. The remaining studies are ones that are sent from a GE archive that are stored as JPEG Lossless. When those JPEG Lossless studies arrive on conquest from the GE archive they are compressed again according to Conquests compression settings in the dicom.ini file. Those studies/images store fine and their file size for the images looks appropriate, but those studies cannot be imported to other workstations and viewed. When a workstation queries Conquest and finds one of these JPEG Lossless studies in question that has arrived from the GE archive, the study transfers successfully and everything looks great until it is viewed and we see this error on the workstations (GE Centricity) 'Data validation for display failed'. This situation only happens to CT exams, all other modalities appear to behave just fine.

My question is, can I use a 'importconverter' rule based on calling AE Title and use compression rules with them? Like when 'AETITLE1' calls I can tell it to store 'as' and when 'AETITLE2' calls I can tell it to store 'j2'. If this is possible would I then need to change my 'IncomingCompression' to 'un' or 'as'?

I will include a copy of my dicom.ini for reference in case any question come up.

Code

# This file contains configuration information for the DICOM server
# Do not edit unless you know what you are doing


[sscscp]
MicroPACS                = sscscp
Edition                  = Personal


# Network configuration: server name and TCP/IP port#
MyACRNema                = RADARCH4
TCPPort                  = 104


# Reference to other files: known dicom servers; database layout; sops
ACRNemaMap               = acrnema.map
kFactorFile              = dicom.sql
SOPClassList             = dgatesop.lst


# Host, database, username and password for MySql database
SQLHost                  = localhost
SQLServer                = conquest
Username                 = root
Password                 = *******
MySql                    = 1
BrowseThroughDBF         = 1
DoubleBackSlashToDB      = 1
UseEscapeStringConstants = 0


# Configure database
TruncateFieldNames       = 10
MaxFieldLength           = 254
MaxFileNameLength        = 255
FixPhilips               = 0
FixKodak                 = 0
KeepAlive                = 0
LargeFileSizeKB          = 1024
PrintSquareLandscape     = 0
UseKpacsDecompression    = 1
ZipTime                  = 05:
UIDPrefix                = 1.2.826.0.1.3680043.2.135.733864.35673984
EnableReadAheadThread    = 1
PatientQuerySortOrder    = 
StudyQuerySortOrder      = 
SeriesQuerySortOrder     = 
ImageQuerySortOrder      = 
EnableComputedFields     = 1
IndexDBF                 = 1
PackDBF                  = 0
LongQueryDBF             = 1000
TCPIPTimeOut             = 300
FailHoldOff              = 60
RetryDelay               = 100
RetryForwardFailed       = 0
ImportExportDragAndDrop  = 0
QueueSize                = 128
WorkListMode             = 0
WorkListReturnsISO_IR_100 = 1
DebugLevel               = 4
Prefetcher               = 0
LRUSort                  = 
AllowTruncate            = 
DecompressNon16BitsJpeg  = 1
UseBuiltInJPEG           = 1
LossyQuality             = 95
IgnoreOutOfMemoryErrors  = 0
PadAEWithZeros           = 0
MAGDeviceFullThreshhold  = 1000
FileNameSyntax           = %studydate\%modality\%id\%sopuid.dcm


# Configuration of compression for incoming images and archival
DroppedFileCompression   = j2
IncomingCompression      = j2
ArchiveCompression       = as


# Names of the database tables
PatientTableName         = DICOMPatients
StudyTableName           = DICOMStudies
SeriesTableName          = DICOMSeries
ImageTableName           = DICOMImages
DMarkTableName           = DICOMAccessUpdates
RegisteredMOPDeviceTable = RegisteredMOPIDs
UIDToMOPIDTable          = UIDToMOPID
UIDToCDRIDTable          = UIDToCDRID


# Banner and host for debug information
PACSName                 = RADARCH4
OperatorConsole          = 127.0.0.1


# Configure email of error messages
MailHost                 = 
MailPort                 = smtp
MailSignon               = 
MailFromName             = 
MailRcptName1            = 
MailCollectTime          = 1
MailWaitTime             = 10


# Configuration of disk(s) to store images
MAGDeviceThreshhold      = 0
IgnoreMAGDeviceThreshold = 0
MAGDevices               = 2
MAGDevice0               = E:\imagedata\images\
MAGDevice1               = f:\imagedata\images\
NightlyCleanThreshhold   = 0
NightlyMoveThreshhold    = 0
NightlyMoveTarget        =

Display More

Thanks for all of your help on this !

marcelvanherk · Sep 13th 2011

Hi,

This may be an issue with 1.4.16 new compression code. Try version 1.4.16e first which has a fix in J2. There is a compression of in importconverter as well which may work for you.

Code

IncomingCompression      = as


[lua]
ImportConverter0 = if Association.Called = 'AE1' then script('compression AS') else script('compression J2');

The compression command works instantaneous, therefore you need to set IncomingCompression to AS.

Marcel

Pacsdude · Sep 14th 2011

Marcel,

We have more than one GE Archive where studies could be sent to Conquest from, so in your code below:

Code

ImportConverter0 = if Association.Called = 'AE1' then script('compression AS') else script('compression J2');

could I use more than one 'Called AE Title' or should I have a ImportConverter for each 'Called AE Title'? Like the following:

Code

ImportConverter0 = if Association.Called = 'AE1', 'AE2' then script('compression AS') else script('compression J2');

OR

Code

ImportConverter0 = if Association.Called = 'AE1' then script('compression AS') else script('compression J2');

Code

ImportConverter1 = if Association.Called = 'AE2' then script('compression AS') else script('compression J2');

Pacsdude · Sep 14th 2011

Marcel,

So, using your new rule shouldn't it be using 'Association.Calling' instead of 'Association.Called'?

Code

# ImportConvertersImportConverter0 = if Association.Calling = 'RADARCH' then script('compression AS') else script('compression J2');ImportConverter1 = if Association.Calling = 'RADARCH3' then script('compression AS') else script('compression J2');

...except it is not compressing any other images from other AE Titles and is storing them uncompressed...because I changed my 'IncomingCompression = as'

[code[RADARCH4] UPACS THREAD 5: STARTED AT: Wed Sep 14 09:03:39 2011
[RADARCH4] Calling Application Title : "CATHROUTE "
[RADARCH4] Called Application Title : "RADARCH4 "
[RADARCH4] Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16384
[RADARCH4] Presentation Context 0 "1.2.840.10008.5.1.4.1.1.1" 1
[RADARCH4] *** Importconverter0.0 error: if Association.Calling = 'RADARCH' then script('compression AS') else script('compression J2')
[RADARCH4] *** Importconverter1.0 error: if Association.Calling = 'RADARCH3' then script('compression AS') else script('compression J2')
[RADARCH4] Written file: f:\imagedata\images\20110914\CR\117603\1.2.392.200036.9107.500.303.4338.20110914.84349.104338.dcm
[RADARCH4] UPACS THREAD 5: ENDED AT: Wed Sep 14 09:03:39 2011
[RADARCH4] UPACS THREAD 5: TOTAL RUNNING TIME: 0 SECONDS
[RADARCH4] *** Queue: holding processing of file f:\imagedata\images\20110914\CR\117603\1.2.392.200036.9107.500.303.4338.20110914.84349.104338.dcm[/code]

I know I'm doing something wrong, can you tell what is wrong in my dicom.ini file?

Code

# This file contains configuration information for the DICOM server
# Do not edit unless you know what you are doing


[sscscp]
MicroPACS                = sscscp
Edition                  = Personal


# Network configuration: server name and TCP/IP port#
MyACRNema                = RADARCH4
TCPPort                  = 104


# Reference to other files: known dicom servers; database layout; sops
ACRNemaMap               = acrnema.map
kFactorFile              = dicom.sql
SOPClassList             = dgatesop.lst


# Host, database, username and password for MySql database
SQLHost                  = localhost
SQLServer                = conquest
Username                 = root
Password                 = ******
MySql                    = 1
BrowseThroughDBF         = 1
DoubleBackSlashToDB      = 1
UseEscapeStringConstants = 0


# Configure database
TruncateFieldNames       = 10
MaxFieldLength           = 254
MaxFileNameLength        = 255
FixPhilips               = 0
FixKodak                 = 0
KeepAlive                = 0
LargeFileSizeKB          = 1024
PrintSquareLandscape     = 0
UseKpacsDecompression    = 1
ZipTime                  = 05:
UIDPrefix                = 1.2.826.0.1.3680043.2.135.733864.35673984
EnableReadAheadThread    = 1
PatientQuerySortOrder    = 
StudyQuerySortOrder      = 
SeriesQuerySortOrder     = 
ImageQuerySortOrder      = 
EnableComputedFields     = 1
IndexDBF                 = 1
PackDBF                  = 0
LongQueryDBF             = 1000
TCPIPTimeOut             = 300
FailHoldOff              = 60
RetryDelay               = 100
RetryForwardFailed       = 0
ImportExportDragAndDrop  = 0
QueueSize                = 128
WorkListMode             = 0
WorkListReturnsISO_IR_100 = 1
DebugLevel               = 4
Prefetcher               = 0
LRUSort                  = 
AllowTruncate            = 
DecompressNon16BitsJpeg  = 1
UseBuiltInJPEG           = 1
LossyQuality             = 95
IgnoreOutOfMemoryErrors  = 0
PadAEWithZeros           = 0
MAGDeviceFullThreshhold  = 1000
FileNameSyntax           = %studydate\%modality\%id\%sopuid.dcm


# Configuration of compression for incoming images and archival
DroppedFileCompression   = j2
IncomingCompression      = as
ArchiveCompression       = as


# Names of the database tables
PatientTableName         = DICOMPatients
StudyTableName           = DICOMStudies
SeriesTableName          = DICOMSeries
ImageTableName           = DICOMImages
DMarkTableName           = DICOMAccessUpdates
RegisteredMOPDeviceTable = RegisteredMOPIDs
UIDToMOPIDTable          = UIDToMOPID
UIDToCDRIDTable          = UIDToCDRID


# Banner and host for debug information
PACSName                 = RADARCH4
OperatorConsole          = 127.0.0.1


# Configure email of error messages
MailHost                 = 
MailPort                 = smtp
MailSignon               = 
MailFromName             = 
MailRcptName1            = 
MailCollectTime          = 1
MailWaitTime             = 10


# Configuration of disk(s) to store images
MAGDeviceThreshhold      = 0
IgnoreMAGDeviceThreshold = 0
MAGDevices               = 2
MAGDevice0               = E:\imagedata\images\
MAGDevice1               = f:\imagedata\images\
NightlyCleanThreshhold   = 0
NightlyMoveThreshhold    = 0
NightlyMoveTarget        = 


# ImportConverters
ImportConverter0 = if Association.Calling = 'RADARCH' then script('compression AS') else script('compression J2');
ImportConverter1 = if Association.Calling = 'RADARCH3' then script('compression AS') else script('compression J2');

Display More

marcelvanherk · Sep 14th 2011

Sorry, use double ==

Code

[lua]
ImportConverter0 = if Association.Calling == 'RADARCH' then script('compression AS') else script('compression J2');
ImportConverter1 = if Association.Calling == 'RADARCH3' then script('compression AS') else script('compression J2');

Marcel

Pacsdude · Sep 15th 2011

marcel,

I tried your code above as:

Code

[lua]ImportConverter0 = if Association.Calling == 'RADARCH' then script('compression AS') else script('compression J2');ImportConverter1 = if Association.Calling == 'RADARCH3' then script('compression AS') else script('compression J2');

and I got back this error:

Code

[RADARCH4] UPACS THREAD 5: STARTED AT: Thu Sep 15 07:49:19 2011
[RADARCH4]  Calling Application Title : "RADARCH         "
[RADARCH4]  Called Application Title : "RADARCH4        "
[RADARCH4]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 0
[RADARCH4]  Presentation Context 0 "1.2.840.10008.5.1.4.1.1.1" 1
[RADARCH4]  Presentation Context 1 "1.2.840.10008.5.1.4.1.1.1" 1
[RADARCH4]  Presentation Context 2 "1.2.840.10008.5.1.4.1.1.1" 1
[RADARCH4] *** lua syntax error [string "if Association.Calling == 'RADARCH' then script('compression AS..."]:1: 'end' expected near '<eof>' in 'if Association.Calling == 'RADARCH' then script('compression AS') else script('compression J2');'
[RADARCH4] *** lua syntax error [string "if Association.Calling == 'RADARCH3' then script('compression A..."]:1: 'end' expected near '<eof>' in 'if Association.Calling == 'RADARCH3' then script('compression AS') else script('compression J2');'
[RADARCH4]

Question... does [lua] need to be on its own line or does it have to be in the dicom.ini at all?

Thanks

marcelvanherk · Sep 15th 2011

Hi,

I indeed forget the 'end' as well. Lua scripts are more user friendly than normal conquest scripts, but the syntax is slightly different from C I am used to. The way to call lua scripts is shown in the lua post on the forum.

Correct statements are (likely to be) - in the lua block:

[lua]
ImportConverter0 = if Association.Calling == 'RADARCH' or Association.Calling == 'RADARCH3' then script('compression AS') else script('compression J2'); end

or outside the [lua] block

ImportConverter0 = lua "if Association.Calling == 'RADARCH' or Association.Calling == 'RADARCH3' then script('compression AS') else script('compression J2'); end"

Marcel

Pacsdude · Sep 15th 2011

Marcel,

Thanks - the code works now!

I'm getting some errors when compressing to J2 on a few select studies that I can't figure out - I was hoping you might have an idea what is happening?

I've noticed this on some XA and some CT exams - here is the log:

Code

9/15/2011 3:15:18 PM [RADARCH4] UPACS THREAD 158: STARTED AT: Thu Sep 15 15:15:18 2011
9/15/2011 3:15:18 PM [RADARCH4] A-ASSOCIATE-RQ Packet Dump
9/15/2011 3:15:18 PM [RADARCH4]  Calling Application Title : "CATHROUTE       "
9/15/2011 3:15:18 PM [RADARCH4]  Called Application Title : "RADARCH4        "
9/15/2011 3:15:18 PM [RADARCH4]  Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16384
9/15/2011 3:15:18 PM [RADARCH4]  Number of Proposed Presentation Contexts: 1
9/15/2011 3:15:18 PM [RADARCH4]  Presentation Context 0 "1.2.840.10008.5.1.4.1.1.12.1" 1
9/15/2011 3:15:18 PM [RADARCH4] Server Command   := 0001
9/15/2011 3:15:18 PM [RADARCH4] Message ID       := 0001
9/15/2011 3:15:18 PM [RADARCH4] 0000,0002     28 UI AffectedSOPClassUID    "1.2.840.10008.5.1.4.1.1.12.1" 
9/15/2011 3:15:18 PM [RADARCH4] 0000,0100      2 US CommandField           1 
9/15/2011 3:15:18 PM [RADARCH4] 0000,0110      2 US MessageID              1 
9/15/2011 3:15:18 PM [RADARCH4] 0000,0700      2 US Priority               0 
9/15/2011 3:15:18 PM [RADARCH4] 0000,0800      2 US DataSetType            258 
9/15/2011 3:15:18 PM [RADARCH4] 0000,1000     56 UI AffectedSOPInstanceU   "1.2.840.113619.2.199.32640.7461.28981.1316092368.2291.73" 
9/15/2011 3:15:18 PM [RADARCH4] 0002,0010     22 UI TransferSyntaxUID      "1.2.840.10008.1.2.4.70" 
9/15/2011 3:15:19 PM [RADARCH4] Importconverter-1.0: compression to J2
9/15/2011 3:15:19 PM [RADARCH4] JPEG compress started.
9/15/2011 3:15:19 PM [RADARCH4] ***[CompressJPEGL]: Could not find the image data.
9/15/2011 3:15:19 PM [RADARCH4] Query Tables: DICOMImages
9/15/2011 3:15:19 PM [RADARCH4] Columns     : ObjectFile, DeviceName
9/15/2011 3:15:19 PM [RADARCH4] Where       : SOPInstanc = '1.2.840.113619.2.199.32640.7461.28981.1316092368.2291.73' AND ImagePat = '8382350'
9/15/2011 3:15:19 PM [RADARCH4] Order       : (null)
9/15/2011 3:15:19 PM [RADARCH4] FreeStore Left 967 on E:\
9/15/2011 3:15:19 PM [RADARCH4] FreeStore Left 16871497 on f:\

Display More

its where it states, ***[CompressJPEGL]: could not find the image data.

The file sizes of these images are usually around 2KB to 6KB - really small and cannot be displayed when imported to a workstation. Give error, 'decompression failed' when the study gets opened on a workstation after being imported.

When I don't use the [lua] script and I go back to 'incomincompression = j2' the file size goes backup to 16MB or so as these are clips from our cath lab.

Thanks

Compression help!

Participate now!

Share