Out of memory error question...

  • I am running into somewhat sporadic out of memory errors in conquest 13b. I do not recall this problem with 12c or earlier. The problem occurs with both of my GE US machines. It seems to happen with 3D/multiframe data sets although the problem is somewhat sporadic ie, several cine or 3D sets will go through, another won't. I get the malloc error message and the server restarts.
    This is actually another problem as the server only restarts if the GUI is running. If not, the service uninstalls, does not start, and I get phone calls that the system is down...
    I am having a hard time trouble shooting this issue as it is somewhat inconsistent. I almost wonder whether this is some sort of hardware issue. But, I did not see problems before v13 so I am left uncertain.
    Right now I am storing with JPEG losless on disk. The US sends uncompressed. I did read the sticky on "out of memory errors" and implemented the suggestion. After that fix, the issue became less frequent but still happens daily.
    Does anyone have experience with a similar problem or ideas?
    Thanks,
    LJJ


    PS Marcel, the 13b patient forwarding is working well as far as I can tell. I have not written you as I am somewhat uncertain if the above problem relates to the patient forwarding. I think I will put the two US machines to a 12c install and see what happens. Hopefully I will get a chance to do this over this weekend. ***Readers, this refers to an unreleased version of Conquest***

  • Hello Marcel


    Thanks for the update. Well, --checklargestmalloc: seems unchanged from 13b. I get 910mb on one machine (4gb installed) and 980mb on an older machine with 2gb memory installed. I will see if things are more stable over this week.
    I also switched to NKI for storage. I have seen some slowdowns in transmission of multiframe US and feel that the problem may lie in JPEG handling of large files. I transfer the 3D raw data along with the images too.
    I will write again later in the week and update you.
    Anything else to know about 14alpha?
    Thanks, LJJ

  • Hi,


    1.4.14alpha has quite a few (as yet undocumented) new export converter options and all (?) bugs fixed that are listed on the 1.4.13 release announcement. It should be as stable as 1.4.13. I am adding delayed delete and a flag to ignore out of memory errors, as it was before 1.4.13.


    Marcel

  • I will use the 14alpha and forward any feedback.
    I have been using a pretty simple forwarding scheme;


    ExportConverter0 = forward patient age -360+0 modality CR to FAIRLIGHT5


    for testing. This has worked quite well. I will have time next week to try some more complex scenarios. If you care to send more documentation of new export options I will try them out.


    I did try to forward patient by modality %m but ran into problems with a couple of CT scanners. They send images one at at time over a fairly slow link (1 minute per image) and I was seeing hundreds of images forwarded by conquest on a 30 image head CT. I think I can minimize the problem by setting the delays and moving the problem machines to a dedicated instance of Conquest...
    Thanks, Leszek

  • Marcel,


    Can you send me conquest 14alpha with the memory error flag disabled? I very much like the patient forwarding features but I am fighting these sporadic out of memory crashes, these are quite disruptive and so far hard to track down.


    Also, for the conquest install at your institution, do you store images compressed? Just curious what kind of working environment you have...
    Thanks, LJJ

  • Hi,


    You can download it at: See: ftp://ftp-rt.nki.nl/outbox/Mar…r/dgate1414alphatest2.zip


    Your new flag is:


    IgnoreOutOfMemoryErrors = 1


    Marcel


    Update history since 1.4.13 release:


    mvh 20071118: Adapted for 64 bits (use SQLLEN for BindField)
    mvh 20071120: Fixed KodakFixer for empty patient ID (crashed on 64 bits only)
    Added [modality xx] and [date yyyymmdd-yyyymmdd] clauses to export [patient|study|series] to AE
    mvh 20071121: Added %V*GGGG,EEEE: search VR in any sequence; version to 1.4.13a
    Added [now -ddd+ddd] and [age -ddd+ddd] clauses to export [patient|study|series] to AE
    No retry on remote dicom error in forward xxx to: would retry zero records response
    Use study date for data, age and now clauses
    mvh 20071123: Made forward command case insensitive; added forward image; added get [patient|study|series|image] from AE
    The latter option makes preretrieve obsolete
    mvh 20071124: Added [sop xxx] clause to [forward|get]
    Reorganized exportconverter code: all converters now do % substitution; reduced code duplication
    Added reading of sequence items like in %V/300c,0060.0/0008,1155 or %V/300c,0060/0008,1155
    Real life example (must all be one one line, max 512 characters):
    ifnotequal "%m", "RTPLAN"; stop; nop "tests modality = RTPLAN";
    ifnotequal "%V*300a,00b2[0,1]", "A2"; stop; nop "test machine name = A2"
    forward to XVI_A2; nop "forwards the current recieved RTPLAN";
    get study modality CT from NKIPACS; nop "collects associated CT";
    get study modality RTSTRUCT sop %V/300c,0060.0/0008,1155 from NKIPACS; nop "collects associated RTSTRUCT";
    forward study modality CT imagetype *AXIAL* to XVI_A2; nop "forwards associated CT";
    forward study modality RTSTRUCT sop %V/300c,0060.0/0008,1155 to XVI_A2; nop "forwards associated RTSTUCT";
    mvh 20071125: Fixed syntax for %V/gggg,eeee.i/*gggg,eeee (search all groups under a specified one)
    mvh 20071126: Start on StorageCommitment
    mvh 20080103: Switched to ms7 compiler; fixed forward and get date, modality, sop filters: would not supress the UIDs
    mvh 20080107: Fixed malformed web viewer header: worked for IE but not for other browsers
    Check presence of dicom.sql in web pages that need it
    Deal with spaces in patient IDs in web pages
    mvh 20080126: Check presence of dgate.dic when starting server
    mvh 20080129: Fix KodakFixer: wrote one byte outside VR data (pr)
    mvh 20080129: Added [imagetype xxxx] clause to [forward|get] (pr)
    mvh 20080130: Small fix in above
    mvh 20080205: Added experimental PadAEWithZeros flag; version to 1.4.14alpha
    mvh 20080210: Added experimental IgnoreOutOfMemoryErrors;
    Added "delete [patient|study|series|image] [date yyyymmdd-yyyymmdd] [now -ddd+ddd] [age -ddd+ddd]
    [modality mm] [sop xxxx] [imagetype xxxx]" export converter

  • Hi Marcel,
    I am running into a lot of new memory out problems with latest Feb10 alpha of conquest 14.
    I thought that jpeg may be the problem so I switched to NKI storage on disk. This is what I get when I try to send a series of scanned films to efilm. So, MAG0 storage is NKI, transmission UN, debug log on:


    [FAIRLIGHT1] MyPatientRootRetrieveGeneric :: RetrieveOn
    [FAIRLIGHT1] Sending file : F:\dicom\CB8269\1.22.333.4444.55555.20071203101357.1.1_0001_000001_12033432312934.v2
    [FAIRLIGHT1] Image Loaded from Read Ahead Thread, returning TRUE
    [FAIRLIGHT1] ***VR:ReAlloc out of memory allocating -536805378 bytes
    [FAIRLIGHT1] MyPatientRootRetrieveGeneric :: RetrieveOn
    [FAIRLIGHT1] Sending file : F:\dicom\CB8269\1.22.333.4444.55555.20071203101357.1.1_0001_000002_12033432312935.v2
    [FAIRLIGHT1] Image Loaded from Read Ahead Thread, returning TRUE
    [FAIRLIGHT1] ***VR:ReAlloc out of memory allocating -536805378 bytes
    [FAIRLIGHT1] MyPatientRootRetrieveGeneric :: RetrieveOn
    [FAIRLIGHT1] Sending file : F:\dicom\CB8269\1.22.333.4444.55555.20080108102240.1.1_0001_000001_12033432312936.v2
    [FAIRLIGHT1] Image Loaded from Read Ahead Thread, returning TRUE
    [FAIRLIGHT1] ***VR:ReAlloc out of memory allocating -536805378 bytes


    So, a set of dicom images is sent and stored by conquest, they are 2.5meg CR images. When I try to push them or pull them to a workstation the above problem occurs. It is confusing as other similar image sets transfer correctly.


    This problem also occurred with CT. Both modalities have been problem free for a year or more with older versions of conquest. I have the Ignoreoutofmemoryerrors = 1 in dicom.ini. I think the problem is with compression and conquest 14alpha Feb10 (although all conquest >12c have been problematic. I will try to store uncompressed on MAG0 and see if it helps. Any comments?


    LJJ

  • Hi Marcel,
    lots of problems with NKI storage on disk and out of memory errors with the Feb10 14alpha and ignorememoryerrors = 1 flag. The images are viewable in Conquest but pulling or pushing them out uncompressed yields out of memory errors. The issue seems to happen with decompression from NKI. I am converting to noncompressed storage until the memory issues get resolved.
    Leszek

  • Marcel,
    I just double checked my dicom.ini in my test conquest and the IgnoreOutOfMemoryErrors = 1 statement is absent. This is the second time that I have been really sure that I put it in and then I look and it is not there. Am I being exceedingly forgetful or can conquest parse/change dicom.ini? I am running 5 instances of conqest for various purposes so I may just have forgotten to change the test version...
    So, now, I am not sure whether the IgnoreOutOfMemoryErrors = 1 flag works or not. I will write again later in the week.
    But, the littleendianexplicit is disabled.
    LJJ

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!