Re: [AMMRL] AMMRL: sporadic ATM failure in ICON-NMR

From: Bob Berno <bberno_at_mcmaster.ca>
Date: Wed, 16 Oct 2024 14:49:07 -0400

Hello Alex,

Yes, we have seen this or a similar error message on one of our spectrometers,
but not on any of the others.  Sometimes it says we don't have a probe installed.
  We have seen it happen in the middle of an overnight automation run,
but we most frequently see it happen when we stop IconNMR to do a nitrogen fill
or perform some other maintenance.  In those instances we usually leave
the IconNMR queue open so that users can add/remove experiments.  Then,
if we try to restart IconNMR, it will fail.  If we instead close IconNMR
and exit TopSpin, then when we restart TopSpin and load the IconNMR queue, it
will run just fine. We saw this error when we were running TopSpin 4.1.4, as
well as now that we are running TopSpin 4.3.0.

I agree with your guess that this has something to do with communication
issues between the workstation and the console.  But exactly what that
is causing this problem, and why it seems to only happen with one
spectrometer, remain a mystery.

One theory I had was that when I upgraded from TopSpin 4.1.1 to 4.1.4,
something got corrupted in the "SpectrOS" software that is installed on
the console.  It was minor enough that the spectrometer mostly worked,
but would throw out these intermittent error messages.

When TopSpin 4.2.0 came out, we upgraded to that.  The error seemed to
come up less often, but was not gone completely.
As soon as we could, we got rid of TopSpin 4.2 and installed v4.3.0.

The error has not gone away, but we can live with it.  Our work around
is that we make sure to stop IconNMR and exit TopSpin at least once per
week on this particular spectrometer.

By contrast, we have another system equipped with a BSNL (no weekly nitrogen fills)
where we have left IconNMR running for months and never had this issue appear. 
We'll pause IconNMR during a helium fill, but then later it starts right back up
again without error.

If you find a solution, I'd love to hear it.

Good luck,

Bob.

On 2024-10-16 12:17 p.m., Alex Greenwood via groups.io wrote:
>
>
> Hi everybody,
>
> Has anybody experienced this before? I have experiments failing
> occasionally at the ATM step in ICON-NMR, followed by all subsequent
> spectra failing at the ATM step. Restarting Topspin will fix the
> problem, and I can re-submit the failed experiment(s). ICON-NMR will
> give an error that reads something along the lines of “Cannot run
> single acquisition: cpp error detected in
> /root/.topspin-BladeEpu/local_acqu/tuneDirs/[datasetname]/1/lists/wobble/pp_drx
> Wobbling error. Details: ATM Failed.”
>
> Sometimes I see a variation on this, where the first experiment will
> complete successfully or almost successfully but with an X under
> “Proc.” In either case there is an error message “Cannot run single
> acquisition: cpp error detected in
> /root/.topspin-BladeEpu/local_acqu/tuneDirs/[datasetname]/1/lists/pp/[pulseprogram]”
> but also “audita.txt file not in data set,” or “/path/to/fid No such>
> file or directory.” In both cases, subsequent experiments fail with ATM errors.
>
> My best guess based on these messages is that this is caused by a
> brief failure in communication between the console (a NEO) and
> computer. Briefly unplugging the ethernet cable during ATM gives a
> similar result (spectrum and subsequent spectra fail, restarting
> Topspin fixes) but with different error messages (something like “ATM
> already in progress”). Replacing the ethernet cable does not stop the
> issue. It’s happening roughly weekly, so manageable but annoying.
>
> My recollection is that this began shortly after updating from Topspin4.1.4
> to 4.3.0, not sure if this is relevant. Any insights are= appreciated!
>
> Alex
>
> Alex Greenwood (he/him)
> Manager, NMR Facility
> University of Cincinnati
> 123C Crosley Hall
> Office: 513-556-9211; Cell: 412-952-8044
> https://www.artsci.uc.edu/departments/chemistry/core-facilities/nuclear-magnetic-resonance-facility.html
>
>

--
Dr. Bob Berno
Manager, NMR Facility
McMaster University, ABB-156
1280 Main St. West
Hamilton, ON L8S 4M1
T: 905-525-9140 x24158 or x24686
---------------------------------------------------------------------------
This email contains CONFIDENTIAL information intended for the specific recipient(s).
If you have an accommodation need for a planned meeting, please e-mail me directly and
I will do my best to make appropriate arrangements.
Should you require any materials sent via this e-mail address in an alternate/accessible
format, please let me know.

McMaster University sits on the traditional Territories of the Mississauga and
Haudenosaunee Nations, and within the lands protected by the "Dish With One
Spoon" wampum agreement.


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#1757): https://urldefense.com/v3/__https://ammrl.groups=
.io/g/main/message/1757__;!!PvDODwlR4mBZyAb0!W7HW9_zB_vcxpfUbpV391xCyQQjMPW=
rsCxIiMdUfw9JFRN7KV7QRdXc60XiTqUDVxLdVGuZvN0Yqo5XL$
Mute This Topic: https://urldefense.com/v3/__https://groups.io/mt/109047127=
/7559972__;!!PvDODwlR4mBZyAb0!W7HW9_zB_vcxpfUbpV391xCyQQjMPWrsCxIiMdUfw9JFR=
N7KV7QRdXc60XiTqUDVxLdVGuZvNyK4mvhE$
Group Owner: main+owner_at_ammrl.groups.io
-=-=-=-=-=-=-=-=-=-=-=-




Received on Wed Oct 16 2024 - 11:49:14 MST

This archive was generated by hypermail 2.4.0 : Fri Nov 01 2024 - 17:08:12 MST