Posts by d.geng

The summary of this issue:

In current setting for our Dicom server Server01

CacheVirtualData         = 1
OverlapVirtualGet        = 0
VirtualServerFor0        = Server02,FIXKODAK,CACHESTUDIES

ForwardCollectDelay               = 120
MaximumExportRetries              = 0
MaximumDelayedFetchForwardRetries = 0

ExportConverters = 1
ExportConverter0                  = ifmatch "%u", "Server03"; stop; between "6", "20"; defer; forward series to Server03

When the daily ExportConverter0 is started, Server01 will do a query on itself to collect the list of scans it needs to forward. Since we enable the ‘virtual’ server, Server01 will query Server02 on IMAGE level. This query on Server02 might take much more time than the query on itself.

dgate.cpp
VirtualQueryCached(DDOPtr, "IMAGE", N, &BDDO);

As the amount of scans for a patient on Server02 grows, the query time increases.

When it reaches the TCPICTimeout, the ExportConverter0 failes. This is the moment when the "***preretrieve/forward xxx to: association lost" is logged.

Then this forward task is listed into the delayed queue and tried over and over again.

Solution:

Remove the virtual server from Server01.

No. But I will my colleague to do so.

Do you know why Conquest has hung up? I could not figure it out.

We also notice in the log file for every 300s (TCPICTimeout), Conquest re-tries.

...
19-08-2020 18:03:21 [Server01]  C-Move Destination: "Server02          "
19-08-2020 18:08:21 [Server01] ***preretrieve/forward xxx to: association lost
...
19-08-2020 18:10:02 [Server01]  C-Move Destination: "Server02          "
19-08-2020 18:14:57 [Server01] Number of Images to send: 251

Thanks for your reply. I think the Delayed Fetch Forward error is related to the following error message in the serverstatus.log

xx-xx-2020 xx:xx:xx [LOCALSERVER] ***preretrieve/forward xxx to: association lost

I can not figure out why it gets this error. I suspect the destination server somehow rejects the coming scan.

My colleague found the following code which might play a role.

    while ( Index < ADDO.GetSize() )
        {
        if (Cancelled)
            {
            ++Failed;
            delete ADDO.Get(Index);
            ++Index;
            continue;
            }
        else
            {
            if (PDU->Link.Poll()) 
                  {
                  OperatorConsole.printf("Recieved cancel request\n", ACRNema);
                  Cancelled = TRUE;
                  }
            }

20100122        mvh     Added experimental C-CANCEL support for all C-MOVES

The Link.Poll() function could indicate whether there is still data coming from the receiver after start the transfer. Why is there always the "cancel request"?

I am not still learning the code.

I get this bug report from my clinical colleague. I am trying to figure out why and help to find a solution. I do not have access to the clinical work station therefore I am not allowed to changed any setting for testing, unfortunately. My colleague needs to know the reason and then can apply the fix.

What is the difference between delayed forward (forward series to) and normal forward (forward to)?

When i use 'between defer forward series to' and 'process series after by', they end up into the same queue. If 'between defer' is on hold, all the elements after it will be blocked. But it still does not explain why after 18:00 in our case the scan is not forwarded to the destination AE.

For 'between defer forward to' it uses a different queue than 'process series after by'. In this case 'process series after by' will always be executed even when 'between defer forward to' is on hold.

Do I understand correctly?

Where can I read more about this queue?

We currently use version 1.4.19d1.

We have updated to this version last year because it is able to count the time after reception of the last image instead of the first image.

We have tested the 'after 3'. It did start counting after reception of the last image.

The code change is logged

Quote

20180301 mvh into_queue_unique resets time stamp for every save; silent destroy2 method

20190205 mvh Made it 1.4.19d1

Recently we have been encountering this error 'delayed fetch forward' during clinical workflow. It blocks 'process after' function in ImportConverter, leading blocking in our clinical workflow.

We define ExportConverter and ImportConverter in dicom.ini as follows. Between 7 and 18 many MR scans are made. They all need to be processed by an executable after 3 seconds when it arrives to this server and backed up to PAK_BACKUP server after 18:00.

ForwardCollectDelay               = 120
MaximumExportRetries              = 0
MaximumDelayedFetchForwardRetries = 0
ExportConverters = 1
ExportConverter0                  = between "6", "20"; defer; forward series to Server03

ImportConverter0                  = process series after 3 by A.exe  "%V0010,0020"

In the DelayedFetchForwardFailures*** file we observe the following two unexpected situations:

1. many scans, made a day early, are still queuing to be forwarded to their destination; They should be forwarded last night after 18:00;
2. 'process series after 3' is still in the queue 'after 3'; they should be triggered 3 seconds after it arrives the server.

We suspect that the queuing function in DGATE, due to the ExportConverter and ImportConverter we introduced, is overflowed.

In dgate.cpp the maximum queue size is defined to 128.

int    QueueSize = 128;                // number of entries in export/copy queue (2k per entry in memory)

Could you guide us to solve the problem? Thanks in advance.

Moreover during the test last week there is one issue from importConverter that i could not re-produce from my local pc. The scan URL ends with (wrong) patient ID.

My setting and log file are:

The ImportConverter is

ImportConverter3 = ifnotequal "%m", "MR"; stop; ifnotmatch "%V0018,1030", "*T2*"; stop; ifnotmatch "%V0008,103E", "cT2_4DTSE*"; process series after 3 by E:\XVIMRL\NO_CRAP_IN_THE_XVIMRL_FOLDERS_PLEASE\ClinicalTestXVIMRL\V1.16\XVIMRL.exe /online "[XVIMRL_B5]:%V0010,0020:%V0010,0020.patient\%V0020,000d.study\%V0020,000e.series\any&ImageType=ORIGINAL.scan"; process series after 3 by E:\XVIMRL\NO_CRAP_IN_THE_XVIMRL_FOLDERS_PLEASE\ClinicalTestXVIMRL\V1.16\XVIMRL.exe /online "[XVIMRL_B5]:%V0010,0020:%V0010,0020.patient\%V0020,000d.study\%V0020,000e.series\any.scan"

The serverstatus for the first patient (ID: F20190807) shows below. The total length (E:\XVIMRL\NO_CRAP_IN_THE_XVIMRL_FOLDERS_PLEASE\ClinicalTestXVIMRL\V1.16\XVIMRL.exe /online "[XVIMRL_B5]:F20190807:F20190807.patient\1.3.46.670589.11.79008.5.0.12924.2019081413174156012.F20190807) is just 194. And the patient ID is in the end of the string which is not expected behavior.

Quote

8/14/2019 1:39:25 PM [XVIMRL_B5] Importconverter3.6: queued process ser - (single object of F20190807) to E:\XVIMRL\NO_CRAP_IN_THE_XVIMRL_FOLDERS_PLEASE\ClinicalTestXVIMRL\V1.16\XVIMRL.exe /online "[XVIMRL_B5]:F20190807:F20190807.patient\1.3.46.670589.11.79008.5.0.12924.2019081413174156012.F20190807

Then I tried another patient (ID: F20190415). This scan URL even ends with the patient ID of the first patient.

Quote

8/14/2019 1:48:11 PM [XVIMRL_B5] Importconverter3.6: queued process ser - (single object of F20190415) to E:\XVIMRL\NO_CRAP_IN_THE_XVIMRL_FOLDERS_PLEASE\ClinicalTestXVIMRL\V1.16\XVIMRL.exe /online "[XVIMRL_B5]:F20190415:F20190415.patient\1.3.46.670589.11.79008.5.0.14000.2019080217022436018.F20190807

Then I changed the importConverter to

ImportConverter3 = ifnotequal "%m", "MR"; stop; ifnotmatch "%V0018,1030", "*T2*"; stop; process series after 3 by E:\XVIMRL\NO_CRAP_IN_THE_XVIMRL_FOLDERS_PLEASE\ClinicalTestXVIMRL\V1.16\XVIMRL.exe /online "[XVIMRL_B5]:%V0010,0020:%V0010,0020.patient\%V0020,000d.study\%V0020,000e.series\any.scan

It works fine.

Quote

8/14/2019 1:55:23 PM [XVIMRL_B5] Importconverter3.4: queued process stu - (single object of F20190807) to E:\XVIMRL\NO_CRAP_IN_THE_XVIMRL_FOLDERS_PLEASE\ClinicalTestXVIMRL\V1.16\XVIMRL.exe /online "[XVIMRL_B5]:F20190807:F20190807.patient\1.3.46.670589.11.79008.5.0.12924.2019081413174156012.study\1.3.46.670589.11.79008.5.0.5452.2019081413361528324.series\any.scan"

On the clinical PC for testing we were only allowed to install our software in a specific folder which has a very long file path.

It was not a good example for using 0-9 repeated 12 times as folder name. It was only handy for me to locate this issue.

It is not an issue for us now and it can be indeed avoided by limiting the file path length.

But it becomes more robust if it supports more characters. Imaging there is an application that needs lots of parameters. Then 360 becomes a constrain.

ImportConverter3 = ifnotequal "%m", "MR"; stop; process study after 3 by F:\MyApp.exe "[DICOM local server]:%V0010,0020:%V0010,0020.patient\%V0020,000d.study\%V0020,000e.series\any&ImageType=ORIGINAL.scan" "V0008,0060" "%V0008,0018" "%V0008,0014" "%V0008,0055" "%V0020,0052" "Vxxxx,yyyy"  

Maybe it should be mentioned in the 'windowsmanual'.

In dicom.ini:

ImportConverter3 = ifnotequal "%m", "MR"; stop; process study after 3 by F:\MRLinac\ClinicalTest\XVIMRLv1.16\012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789\XVIMRL.exe "[DICOM local server]:%V0010,0020:%V0010,0020.patient\%V0020,000d.study\%V0020,000e.series\any&ImageType=ORIGINAL.scan"

In the serverstatus.log from Conquest DICOM server:

8/14/2019 1:39:25 PM [XVIMRL_B5] Importconverter3.6: queued process ser - (single object of F20190807) to F:\MRLinac\ClinicalTest\XVIMRLv1.16\012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789\XVIMRL.exe "[DICOM local server]:F180515A:F180515A.patient\1.3.46.670589.11.79008.5.0.10628.2019041716163908010.study\1.3.46.670589.11.79008.5.0.7260.2019041717033285678.series\any&ImageType=ORIGINAL.sca"
...

The total length(

F:\MRLinac\ClinicalTest\XVIMRLv1.16\012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789\XVIMRL.exe "[DICOM local server]:F180515A:F180515A.patient\1.3.46.670589.11.79008.5.0.10628.2019041716163908010.study\1.3.46.670589.11.79008.5.0.7260.2019041717033285678.series\any&ImageType=ORIGINAL.sca"

) is 360.

Anything after character 360 will be removed.

It would be cool to support more characters.