AMMRL: Summary of User Competency and Training Survey

From: Eric Paulson <Eric.Paulson_at_yale.edu>
Date: Tue, 20 Jun 2006 16:03:45 -0400

Thanks to everyone who responded to the survey.

As one person put it, a problem with surveys such as this is that they tend
to focus on averages and the majority, which might imply that these
"average" results are the "right" thing to be doing. While I will try
generalize and summarize where it seems appropriate, I think it is often
more interesting and informative to focus on the variety of answers than
the average.


FACILITIES
1.) In what type of institution is your facility?
I received responses from 59 academic (82%) and 13 industrial (18%)
institutions. The rest of my summary will assume that the responses I
received are representative of our community, which may or may not be
true.

The majority of institutions (70%) had a "large" (>50) pool of users.

The smaller places were a more diverse group, including undergraduate-only
institutions, specialty (i.e. solids or biochemistry) NMR facilities and
smaller companies. My survey questions clearly made more sense to those
with a large pool of users, but I really appreciate these others taking
the time to contribute as well.

2.) Do you use automation for your NMR spectrometers?

40% of academic institutions were using automation (I should have been more
clear, but most interpreted this to mean "sample changers.") Of this 40%,
most had fewer than 1/3 of their instruments automated. Only in 7% of
academic institutions were the majority of instruments automated.

This is a contrast to industry, where 92% use automation, and 62% had *all*
of their instruments automated.

Automation definitely seems to improve sample throughput at both academic
and industrial institutions, as most reported that the automated
instruments carried a large share of all experiments. A few places had
automation but didn't really use it.

Of the academic institutions without automation, about 10% said they don't
wish to have any automation as it is important for users to have a more
hands-on experience, and about 10% said that they were thinking or
planning on automating some of their instruments in the future.

3.) Does your facility run NMR experiments as a "service" for users?

The uniformity of answers here surprised me. More than 90% of academic
institutions do little or no "service" work (I should have been more clear
here as well, most correctly interpreted this to mean "running experiments
for users"). However some of those 10% of academic institutions run a
significant fraction (>25 %) of their samples as a service.

In industry, it seems that virtually all NMR experiments are run either by
the end user in automated mode, or by NMR experts as a service. Some
exceptions to this are at companies with a small number of users. At some
places the service extends to structural characterization -- the end user
does not interpret spectra.

4.) How many spectrometers are in your facility? About what fraction of all
available spectrometer time goes unused?

In academia, the number of spectrometers ranged from 1 to 18 with an
average of 4.8. The fraction of unused time ranged from 5% to 99% with an
average of 39%.

In industry, the spectrometers ranged from 1 to 10 with an average of 3.2.
The unused time ranged from 10% to 65% with an average of 37%.

A few other interesting statistics (average in parenthesis):
Number of users/spectrometer in academia: 2 - 58 (22.5)
Number of users/spectrometer in industry: 1 - 35 (15.7)
Number of users/staff in academia: 7 - 230 (63.2)
Number of users/staff in industry: 1 - 160 (41.7)
Number of spectrometers/staff in academia: 0.5 - 8 (2.88)
Number of spectrometer/staff in industry: 1 - 10 (2.43)

Take these numbers with a grain of salt, not everybody defined users and
staff in the same way.

There was *not* a correlation between the number of users/spectrometer and
percentage of unused time, even at the "large" institutions (somewhat
surprising to me). I think this is a clear demonstration that although we
are all "magnetic resonance laboratories" the users and the types of
experiments used in these labs are quite diverse, and so it is hard to
draw any conclusions based on "averages."


5.) How many people support the NMR spectrometers? About how much of their
time is spent doing training?

The number of staff vary from 0.5 to 5 in academia and 1 to 4 in industry,
somewhat correlated with the number of spectrometers.

I didn't tabulate the time usage, because people chose to answer it in
different ways, but I would estimate that academic staff spend 5-20% of
their time doing training, and industrial staff use 0-5% of their time
doing training.

6.) About how many users do you have?
Number of academic users ranged from 7 - 300 with an average of 109 and a
standard deviation of 75.
Number of industrial users ranged from 2 - 160 with an average of 57 and a
standard deviation of 56.


TRAINING METHODS

My primary interest in conducting this survey was to get new ideas for
training methods and learn what works best. I don't know how much I
succeeded in this goal. I was actually surprised how many places take
basically the same approach to training -- a 5 minute to 6 (or more) hour
one-on-one (or maybe two- to three- on one) "checkout" for running the
spectrometer. My guess is that most everyone takes this approach because
new users show up at a wide variety of times and they all want to get
started taking data as quickly as possible. Also, as one responder wrote,
for this sort of thing people must "learn by doing."

There were, however, some interesting variations and alternate approaches
which I will try to enumerate below.

1.) Do you have different distinct groups of users with respect to
training, and if so, what are they? (If you do, you can break down your
answers to later questions based on these groups)

The answers to this varied wildly, of course. Some places distinguish
undergraduates from graduate students/postdocs or students from faculty.
Some places have certain research groups that do all their training
internally. Others have formally specified different "levels" of use for
granting access to different instruments, generally graduating from
low-MHz "walkup" machines to high-field "research" machines. Some break
down their users by research type, such as: organic (H+C only), inorganic
(other nuclei), biomolecular (aqueous samples, complex multidimensional
experiments), solids, imagers. Some have a "classroom track" and "others."
Many break down their users by their level of interest: walkup only, those
who want more advanced experiments or variable temperatures, etc. One
broke them down by ethnicity, but he/she was only joking. :)

Most said "no" meaning either all their users were the same or that they
were all so different that that had to be treated on a case-by-case basis.


2.) Broadly speaking, what does a typical new user (or each type of typical
new user) already know about NMR before you train them?

Here again, of course, most answers varied quite a lot, although most seem
to already know the basics of interpreting proton spectra (probably from
undergraduate organic chemistry) before they come to the NMR facility. Few
have learned much about multidimensional NMR, something that could likely
best be covered in a classroom setting (at least initially).

3.) Describe the typical training procedure for a new user of your
facility.

Most places do a one-on-one (or small group) training by a facility staff
member while sitting at the spectrometer. A few places have a formal
course with a significant "hands-on" component that must be taken before
access is granted. Some places allow initial training by an experienced
user, usually with some sort of "checkout" by a staff member.

A few interesting ideas/variations are:
First a student must read a written guide/manual, then come back for
one-on-one training at the spectrometer.
First a student must attend a lecture followed by a tour, then have
training at the spectrometer.
First a student learns how to collect data at the spectrometer, then a
student learns how to do processing/plotting on an offline computer.
The reverse of above: first a student does processing/plotting of a data
set on an offline computer, then learns how to collect data at the
spectrometer.
A group of students is shown a demo of spectrometer usage before training
at the spectrometer.
A student is given training materials and must give a verbal presentation
on NMR basics before being trained at the spectrometer.
A student watches an online training video before being trained at the
spectrometer.

4.) How much time does this typical training procedure take?

This certainly varied. The least was "about 5 minutes" in an industrial
facility for training on autosampler usage. The most was difficult to
tell, as you could count multiple hours of classroom time, multiple days
of one-on-one training, etc. I believe the most common response was about
2 hours of one-on-one training.

5.) Does the user pay for instrument use during training, and/or are there
other charges related to training?

This was pretty evenly divided between the user being charged standard
rates during training and the user being charged nothing during training.
Very few places charged anything extra for training. Some charged the
"service rate" rather than the "self-use rate" during training.

6.) Who does the majority of user training? (i.e. NMR facility director,
teaching assistants, highly trained users, other specific faculty/staff
members)

At most places, the facility director does the majority of user training.
Larger institutions with mutiple NMR staff had more variable answers here.

7.) Is there some sort of competency test required before access is granted
to the spectrometers? If so, what is it?

The most common response was "no." I believe most of the "no" answers were
also places that used one-on-one training by a staff member. Some places
used a written test or required the user to demonstrate that they could
successfully acquire data while a staff member observed.


8.) Who has the final say over when or how a new user is granted access to
the facility?

Most commonly, the facility director had this authority. Some responders
expressed frustration that faculty would "override" their decisions about
the amount of training or other aspects of user access to the facility.

9.) If not addressed in questions 1-7, what are the procedures for getting
access to more "advanced" methods? (i.e. variable temperatures, "bigger"
magnets, other nuclei, solid state NMR, etc.)

At most places, the response was along the lines of "ask and ye shall
receive" (after some additional training). Others had different "levels"
of users and a formal method for progressing to more advanced levels.

At least one person talked about "advertising" some of the more advanced
experiments that are possible by NMR in the form of short presentations
(or by other means). This struck me as a good thing to do, especially if
the burden is on the users to ask for such experiments. However, I suppose
it could have a downside if too many users were encouraged to ask for
advanced experiments they don't really need.

10.) Is NMR covered in a class or various classes at your institution, and
how does this fit into new user training? Who teaches these classes?

At some places, the "classroom NMR" (if any) is covered as part of a class,
often organic chemistry and/or analytical chemistry, taught by a faculty
member unrelated to the NMR facility. It seems that at a lot of these
places, the content of these courses and the training in the NMR facility
were not closely interconnected, but this is me "reading between the
lines." At other places, the individual or individuals who taught
NMR-related classes were also responsible for the NMR spectrometers, so
presumably the classroom and "hands-on" NMR training has more continuity.

11.) What types of training materials are used in NMR training? (e.g. web
pages, textbooks, printed handouts, etc.)
12.) Have you already been, or would you be, interested in pooling your
training resources with the AMMRL community? If so, how and what?

I have tried to list below the links to websites and other offers to share
training materials (or advice on what to use). Hopefully I did not miss
anybody. Also, I hope I did not include anyone who wished to remain
anonymous.

Phil Dennison:
http://www.chem.uci.edu/research/facilities/nmr/newuser.shtml

Carlos Pacheco:
http://www.princeton.edu/~nmr/material.htm

Rainer Haessner (in German):
http://www.org.chemie.tu-muenchen.de/people/rh/ammrl/aufgabe.pdf
http://www.org.chemie.tu-muenchen.de/people/rh/ammrl/loesung.pdf

Steven Graham:
Offered a manual section on acquiring 1H spectra.

Jürgen Schulte:
Online Manuals (for Bruker AM/AC with or without Tecmag upgrade):
http://www.chem.binghamton.edu/staff/schulte/AM-AC-Manual.doc
http://www.chem.binghamton.edu/staff/schulte/AM-Tecmag-Manual.doc
NMR Course:
http://www.chem.binghamton.edu/staff/schulte/CHEM585f/Chem585f-Bruker.doc
http://www.chem.binghamton.edu/staff/schulte/CHEM585f/Chem585f-Tecmag.doc

Russell Hopson:
Has PDF files for those switching from XWINNMR to TOPSPIN.

Brian Breczinski pointed out the following online book by Joseph P. Hornak:
http://www.cis.rit.edu/htbooks/nmr/

Jeff Simpson:
Suggests organizing training workshops.

Jeff Ellena:
Instructions for Varian (VNMR 6.1) and GE GN and QE 300 (with Tecmag
acquisition system) at http://ernst.chem.virginia.edu

Dave Scott:
http://www.cif.iastate.edu/NMR/training.shtml

Keith Burke:
http://www.chemistry.nmsu.edu/Instrumentation/NMR_page.html

Alan Kenwright:
Offered to share interpretation examples and problems.

Thomas Stringfellow:
http://www.pharmacy.wisc.edu/AIC/NMR/UserNotes/usernotes.cfm

Beverly Ostrowski:
Offers a 3-5 day NMR workshop several times per year. Is willing to share
online documents.

Yuyang Wu:
http://pyrite.chem.northwestern.edu/analyticalserviceslab/NMR/manuals.htm

Jonathan Foglein:
Offered a Varian instruction sheet.

Debbie Shalev:
Willing to contribute lecture presentations.

Jaroslav Zajicek:
Offered to participate in writing unified handouts.


Several people suggested that a wiki would be useful for this. To help
start this, I added a rudimentary wiki page on training materials at the
ammrl how-to wiki:
http://129.187.121.8/ammrl-wiki-status/index.php/HowTo

Hopefully others will revise & improve this.


TRAINING TOPICS
Most reported that they focus primarily on "How the instrument works",
"Basic data collection" and "Output." Some also cover "Basic NMR Theory."
Few cover "Spectral Interpretation" or "More advanced topics." If people
are interested in a more quantative or otherwise detailed analysis of
training topics, please ask me.

USER ISSUES
By and large we are very lucky with regards to our user base. Many reported
that they have very few problems with their users. Of the issues that
people reported, more had problems with users breaking the rules with
regards to access and training than with users breaking equipment. Very
few have ever banned a user, and most of those have only banned one or two
users. More common "punishments," if any, were: talking to the user,
talking to their supervisor or PI, banning from a specific instrument, and
making the user (or their advisor) pay for damages. Many NMR managers were
unconcerned whether or not their users were collecting "good data," but a
few expressed that this was a big concern.

As far as what we as a community should ask of our vendors to make training
users easier, the most common answer, if any, was "better manuals" or
training materials, especially for non-NMR experts. Some suggested that we
as a community could provide specifics for improving the vendor's manuals.
Others asked for more consitency in the software commands and interface
between different software versions (or even vendors). A few suggested
that they could implement changes to make it more difficult to 1.) drop
the sample depth gauge into the spectrometer and 2.) drop a second sample
on top of one already in the magnet.

Thanks again to everyone who participated in the survey.

-- 
Eric Paulson
NMR Manager
Chemical Instrumentation Center
Yale University
225 Prospect St.
New Haven, CT 06511
203-432-3942
Received on Wed Jun 21 2006 - 08:39:18 MST

This archive was generated by hypermail 2.4.0 : Sun Jun 11 2023 - 14:36:45 MST