Error Invalid element length

  • Hi,
    I'm using Conquest 1.4.16rc4 and I've created a folder that Conquest monitors for incoming images.
    Recently I tried changing some DICOM tags using a .cq file in order to modify some tags.
    Lately I received an error while trying to upload some images. It says:

    Code
    10/03/2011 13:56:33 [CONQUEST2] ***Encountered an invalid element length during load of DCM file (in 7fe00010)
    10/03/2011 13:56:33 [CONQUEST2] ***Length = 850368
    10/03/2011 13:56:33 [CONQUEST2] ***Group Length = 12
    10/03/2011 13:56:33 [CONQUEST2] ***[AddImageFile] E:\ConquestNNT\data\incoming\45if01fa3000.dcm -FAILED: Error on Load


    Using a hex editor I could see that the value in DICOM tag 7fe00000 (PixelDataGroupLength) is wrong. It actually contains the value 12, while in reality it should be much larger.
    I know that it is fixed when I change some DICOM tags using DCMTK, but can this somehow automatically be fixed by Conquest while importing? I would like to avoid using an extra tool.
    Regard,
    Johan

  • Hi Johan


    these tests are there to avoid conquest crashing on attempting to load invalid data. I can make the check a bit less rigorous (it already accepts 0 group length, but I am not sure what limit I should use then.


    Marcel

  • Hi Marcel,
    When you speak about tests, is this something we can change ourselves or is it in the compiled dgate-code? I guess there is no way to let Conquest behave like DCMTK which apparently corrects the invalid group length when modifying some other DICOM tag?
    Johan

  • Hi,
    I had a look at the trnsyn.cxx source code and I saw there is already a fix for the "Bad Bad Novarod" :wink:
    The dicomfiles are coming from a NewTom 5G Conebeam CT reported as NT5G in DICOM tag 0008,1090.
    Tag 7FE0,0000 contains value 12 and tag 7FE0,0010 contains the actual pixel data length. Tag 7FE,0000 should contain the pixel data length + 12.
    I guess the code was written in C++ but I never did anything in this language before.
    Can you give me a hint as to where I should start in order to make a fix for this problem?


    Johan

  • Hi Johan,


    I am not sure if and how to solve this: either conquest does some checks and rejects faulty data, or faulty data could cause it to crash. One option may be to add a flag to allow it to accept any data without checks.


    Marcel

  • Hi Marcel,
    I don't know how they do it, but DCMTK somehow corrects the content of tag 7fe0,000 when some other tag is modified.
    I guess they check the content of this tag.
    As it always has the value of pixel data - value element length ( 7ef0,0010) + 12 one could imagine an option where in the ini file one can set a flag forcing conquest to correct 7fe0,0000 upon adding a new file.
    Idon't know however if this is possible.

  • Hi Marcel,
    Apparently the statement in my last post was not correct: 7fe0,0000 is not always 7fe0 + 12. I just saw a case where the difference was only 8. So please disregard my previous post.
    On the dcmtk forum however I saw the following statement: "If you think of automatic group length or item length correction, the answer is yes. The dcmdata routines automatically recalculate such values when writing the file. " ( http://forum.dcmtk.org/viewtop…ight=automatic+correction)
    So an option in the dicom.ini file that would force such a recalculation in Conquest could be the solution to my problem.

  • Hi,


    conquest automatically recalculates the lenghts, so if the length is zero all will go allright. It is just that conquest does not accept a non-zero invalid length as the lenght is used as a safeguard during parsing the dicom stream.


    I consider added a setting NoDICOMCHECK to disable all checking. This will cause conquest to crash on invalid files (as with older versions like 1.4.14), but allows reading your files. As a temp solution you could use an older version of conquest (you can access all versions with ftp) to load the files and transmit them to a newer release.


    Marcel

  • Hi Marcel,
    Using a prior version of Conquest would be a solution, but I need the "incoming directory" monitoring from version 1.4.16. The problem is that we could send the files directly from the modality but processing and sending them using DCMTK took us 12 min for 650 files, thus blocking the modality. So I came up with a combined DCMTK - Conquest solution and this reduced the total time to about 5 min, but the modality itself is free for the next patient after only 1 min 30. The remaining time is spent on a separate server.
    Anyway, I guess I will have to wait for the NoDICOMCHECK option. I presume lengths will be recalculated even when this option is set?
    Johan

  • Hi,
    I'm happy to confirm that the "Nodicomcheck" option works as expected: The dicom files with the invalid element length are accepted by Conquest without any problems when the option is set. The error itself is corrected by Conquest;


    Johan

  • Hi,


    we have a problem with some MR stydies: our Syngo RT division of workstations doesn't accept some series at network level whereas diagnostic workstations (Centricity RA600), clinic pacs, and Conquest don't have any problem with them.
    It's been investigated, that the 6000,xxxx overlay plane module generated by Syngo MR is the cause.


    In Conquest 1.4.16 this part dumps to


    Code
    (6000,0010) UN \c0\01 # 2 ?(6000,0011) UN \c0\01 # 2 ?(6000,0015) UN [1 ] # 2 ?(6000,0022) UN [Siemens MedCom Object Graphics] # 30 ?(6000,0040) UN [G ] # 2 ?(6000,0045) UN [USER] # 4 ?(6000,0050) UN \01\00\01\00 # 4 ?(6000,0051) UN \01\00 # 2 ?(6000,0100) UN \01\00 # 2 ?(6000,0102) UN \00\00 # 2 ?(6000,3000) UN (not loaded) # 25202 ?


    more clear from dcmtk:



    Size of overlay data should be 448 x 448 x 1bit = 25088byte
    In every image of the same dimension 448 x 448 the size of overlay data is different but too big.
    Is there a way to simply resize tag 6000,3000?
    How can I adress unknown tags numerical in lua (like %V6000,0010 in traditional Conquest converter scripts)?
    We don't really need the overlay. How could I remove all 6000,xxxx tags without addressing each tag individually?


    Thanks in advance


    Gerald

  • Hi,


    lua is they way to go to delete these items. There is no option to resize.


    In lua you can loop and address tags as follows:


    Data['6000,3000']


    All string facilities can be used to create the '' embedded string. But not that it is a string and conquest expects 4 hex digits.


    Marcel

  • Hi,


    access to tag 6000,3000 seems not to work correctly. Function testfn() called from importconverter0


    Code
    function testfn() print('Type : ',type(Data['6000,3000'])) print('Length : ',#Data['6000,3000']) print(Data['6000,3000']) Data['6000,3000']=string.sub(Data['6000,3000'],1,25088)end


    produced the following output to serverlog


    Code
    [CQDCM1416] Type : string[CQDCM1416] Length : 10[CQDCM1416] (too long)[CQDCM1416] Importconverter-1.0 executes: set 6000,3000 to "(too long)"


    And exactly


    Code
    (6000,3000) UN [(too long)] # 10 ?


    is the new content of tag 6000,3000. But I can remove it when I assign nil.
    Is there another lua binding for large binary VRs necessary? If so, please put it on the wishlist. Lua integration in Conquest is great!


    Gerald

  • Hi,


    there is no binding yet for large VR's -- (too long) is coded in to avoid access violations. So you can use lua to test if any VR exists, and you can delete the tag. How would you propose to code access to long VR's in lua? Lua string can be arbitrary long, but the conquest lua and script infrastructure is not well geared to very long strings (even if you can read it, you cannot set it).


    Maybe something like Data.GetVR(group, element) which returns a table, and Data.SetVR(group, element, table) to write a VR.


    Marcel

  • Hi,


    lua command setvr(object, g, e, data), getvr(object, g, e) and Data:SetVR(g,e,data) and Data:GetVR(g,e) are in 1.4.16a just released. Work both for any normal VR (returns/uses a table wigth elements from 0..vr->Length-1) and for sequences (returns/uses a dicomobjectarray object).


    For testing I used:


    dgate "--lua:x=newdicomobject();x:Read(file);y=x:GetVR(0x7fe0,0x10);x:SetVR(100,100,y);x:Write('c:\\t.dcm')"


    Marcel

  • Hi,


    I played a bit with lua and overlay. This is the relevant part of the GUI header dump

    Code
    (6000,0000) UN \80\80\00\00 # 4 ?(6000,0010) UN \00\02 # 2 ?(6000,0011) UN \00\02 # 2 ?(6000,0015) UN [1 ] # 2 ?(6000,0022) UN [Siemens MedCom Object Graphics] # 30 ?(6000,0040) UN [G ] # 2 ?(6000,0050) UN \01\00\01\00 # 4 ?(6000,0051) UN \01\00 # 2 ?(6000,0100) UN \01\00 # 2 ?(6000,0102) UN \00\00 # 2 ?(6000,3000) UN (not loaded) # 32768 ?


    Because I got problems calculating the right overlay size, I run this for the tags above

    Code
    VR='6000,0010'print(VR,' type:',type(Data[VR]),' = ',Data[VR])


    and this is the (cleaned) output

    Code
    6000,0000 type: nil = nil6000,0010 type: nil = nil6000,0011 type: nil = nil6000,0015 type: string = 16000,0022 type: string = Siemens MedCom Object Graphics6000,0040 type: string = G6000,0050 type: string = 6000,0051 type: string = 6000,0100 type: string = 6000,0102 type: nil = nil


    every VR starting with a null character is of type nil and I don't know how to get the expected value. Tags 6000,0050 and 6000,0051 consists of only one character with code 1.
    It looks like the method above works for string-like VR, but not for binary data.


    At next I tried the new command GetVR() for the binary values

    Code
    od=Data:GetVR(0x6000,0x0010)print(#od)for i=-1,5 do print(od[i]) end


    This is the cleaned output

    Code
    1nil02nilnilnilnil


    and for tag 6000,0050

    Code
    3
    nil
    1
    0
    1
    0
    nil
    nil


    This works well but I'm a bit confused about #od beeing one less than the real table size.
    I don't know lua very well, but I assume, that this is because the first table element has the index [0].


    Btw. the result dicomobjectarray of the "dicomquery()" command starts at [0], and the result of "dbquery()" at [1][1].
    May be this is not relevant when the lua iterators pairs() or ipairs() are used.


    Gerald

  • Hi Gerald,


    Indeed, Data.Item maps all VR's to strings, except 16 and 32 bits integers. There is no distinction between an empty VR ans a missing VR. Only the new GetVR will work on binary data: it will only return nil if the VR does not exists. It will return a table for any nornal VR and it will return a userdata for a sequence, that can then be addressed as od[N].Tag.


    Correct! As you see I am a bit torn between lua definition (starts at 1) and C (starts at 0). In any case, #od will give you the element number of the last element. I think that you can shorten a VR as follows:


    od = Data:GetVR(0x6000,0x0010);
    print('actual length = ', #od+1);
    od[correctlength-1] = nil;
    Data:SetVR(0x6000,0x0010, od);


    Marcel

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!