Difference between revisions of "Parser JSmol"

From NMReDATA
Jump to: navigation, search
(recording features that are implemented in the reader page using JSmol)
(recording features that are implemented in the reader page using JSmol)
Line 142: Line 142:
 
[[File:Data-parser.png]] Equivalent spins, with the keyword "Equivalent=".
 
[[File:Data-parser.png]] Equivalent spins, with the keyword "Equivalent=".
 
* Pre-implemented: it is not displayed, but it is detected and a warning alert is displayed (with a count).
 
* Pre-implemented: it is not displayed, but it is detected and a warning alert is displayed (with a count).
 +
 +
When there are two structures (e.g. 2D and 3D) in the same SDF file, we might need one ASSIGNMENT tag per structure.
 +
* If the same atom labels are in both models, maybe just one ASSIGNMENT tag is valid. ''This needs investigation.''
 +
* They may also be different with respect to the "Interchangeable=" lines.
 +
  
 
=== <NMREDATA_J> (couplings) ===
 
=== <NMREDATA_J> (couplings) ===
Line 192: Line 197:
  
 
== Generation of a 3D molecular structure ==
 
== Generation of a 3D molecular structure ==
 +
 +
[[File:Data-parser.png]]
 +
Two methods are currently available for calculating a reasonable 3D structure.
 +
They also involve adding the implicit hydrogens which are not included in the original 2D structure.
 +
* Using JSmol capabilities.
 +
* Sending the structure to the ALATIS server and retrieving the 3D structure data.
 +
 +
Caveats:
 +
* Getting the proper stereo configuration is not possible in an automated way. We need a wizard manually assisted and validated by the user. ''Implementation has started for this but it still needs much work.''
 +
* The added atoms should be labelled in accordance with the data in the ASSIGNMENT tag or spectrum tags.
 +
* Ensure the addition of all H's in a single operation (*).
 +
 +
(*)The simplest JSmol method fails to add all H's to some flat models. Two addition steps with a 3D optimisation in between are needed.
 +
 +
So we are now testing 2 enhanced methods:
 +
# Random push of the C's out of the z=0 plane + add hydrogens + short optimisation + add hydrogens again + full optimisation.
 +
# Find C's with 3 bonds to heavy atoms (these atoms seem to be those with *problems in the original simple method) + attach a new hydrogen to them, positioned out of the z=0 plane + run a full optimisation.
 +
 +
Testing is need to decide which method is better.
 +
 +
In both cases, further runs of optimisation may be necessary, driven by the user with the aid of numerical reports of the drop in energy (convergence of he minimisation).
 +
 +
Addition of H's to CR<sub>2</sub> is particularly problematic:
 +
* No problem when the two H are isochronous and the two H's will receive the same label with and without primes. The option should be offered to add "a"/"b" labels, or else just allow edition of the labels.
 +
* When the two H are not isochronous, user intervention is required to assign them properly or, if the answer is unknown, to  use the "Interchangeable=label1, labels\" line in the ASSIGNMENT tag. Possibly four options:
 +
*# correct
 +
*# reverse (flip the two labels)
 +
*# ignore (write no "interchangeable" line)
 +
*# flag labels as Interchangeable
 +
We could have a link to provide some additional explanation in a wiki page. If this text could be in a separate file, it could be edited easily without changing the "code" part of the page. (Nice for updates - this is not necessary.)
 +
 +
Desirable: to display in a different style the bonds with unknown chirality.
 +
 +
Question: is it necessary to modify the 2D file according to the result of the 3D model and its labeling?

Revision as of 17:26, 12 May 2019

Parser for NMReDATA zipfiles in a web page using JSmol

This is a copy of contents in Parser and portions of NMReDATA tag format with added annotations of what is being implemented in the NMReDATA file reader, HTML application using JSmol. It also includes the result of email discussions.

Reading the packaged contents

Data-parser-ok.png The content of the input is kept in memory so that it can be written later.

Data-parser-ok.png Files contained in the zipfile are listed in a folder and file tree on the left column, alphabetically sorted. The tree is interactive in order to display contents of those files.

Data-parser-ok.png In the case of file not found (filename called from a button) or non-zip format, a warning is displayed.

Reading the SDF file

Data-parser.png (For debugging) The full contents of the sdf file may be examined via the right-click menu on the file tree.

Several models

The SDF file may contain more than one structure, each one (molblock) with its own tags. We expect that the NMReDATA tags will be included only on the first model, but it might be otherwise.

Data-parser-ok.png All tags for all models are displayed in the central section, first the group of NMReDATA tags and then the group with all other tags. If needed, a number is added in front to identify the model number.

The first model will usually be 2D. It is desirable to be 3D but this will not always happen.

We expect that a second model might be the 3D structure.

Data-parser-ok.png All structure models are transferred to the JSmol panel, and displayed there one at a time. Buttons are added below to switch the display from one model to another. The 2D or 3D nature is detected using two methods: the code in line 2 of the molblock (characters 21,22, according to the MOL/SDF V2000 format specification) and the Z coordinates of all atoms. The first takes precedence, but any discrepancies raise a warning.

It is important to keep any stereochemical information. Even in 2D data the atoms may have extra information, i.e. for explicit hydrogen atoms the chirality is given by the fourth column of the bond list in the molblock (more precisely, characters 10 to 12, according to the MOL/SDF V2000 format specification).

Data-parser-ok.png Explicit H atoms with stereo information in their bond are coloured light salmon (bisque, bond up) or light blue (powderBlue, bond down) in the JSmol structure. They are also moved 0.4Å up or down in the Z axis direction for better viewing.

Tags

The SDF files may contain diverse tags (not only the NMREDATA tags). Be ready for the tags to appear in any order.

Data-parser-ok.png They are accepted in any order and are displayed as they arrive, except that NMREDATA tags are grouped first. (We might sort them if needed)

All tags should be written in the output SDF file. We recommend to write tags then in the same order.

Determine how to read the NMREDATA tags

Scan the tags and list the index of the ones including NMREDATA_ in their name. Read the NMReDATA tags.

Data-parser-ok.png Both NMReDATA tags and any others are displayed, with their contents.

Data-parser-ok.png Keep in mind the end of line problem.

  1. If backslash+<EOL> pair is present, it is replaced with <EOL> for display.
  2. If backslash only is present, it is replaced with <EOL> for display.
  3. When saving, backslash will have to be added before any <EOL>.
  4. There seems to be no need to check the value of VERSION (although it is possible).

Read the NMREDATA tags

Many simple tags have no particular format. (NMREDATA_SOLVENT, NMREDATA_VERSION, etc.)

But most "complex" tags (NMREDATA_ASSIGNMENT, NMREDATA_J, NMREDATA_1H, etc.) all have a common general structure:

Two type of lines should be distinguished:

  • Property lines
  • Item from a list

The "property lines" contain a serie of characters (letters) followed the "=" sign followed by the value of the variable. They can be directly used as name and value of object properties. The value should be read as text , because they have to be revritten later when writing the .sdf file ...

The "property lines" should be identified as such to distinguish them from element of a list. Property lines should be located before the list, but some may follow the list (not recommended, but possible).

IMPORTANT: Note that a property may appear more than once. In this case, it should be read into an array (and be written in the same order when writing the file).

The "Item from a list" have a format that depends on the tag. We have four types of tags:

  • NMREDATA_ASSIGNMENT
  • NMREDATA_J
  • NMREDATA_1D_ ...
  • NMREDATA_2D_ ...

For NMREDATA_1D_ and NMREDATA_2D_ see 1D_attributes and 2D_attributes.

We recommend the store the list as an array of array of characters, and analyse it later. We suggest that when analysing a tag, each line is tested to see if it is a "property line" (see the format above). If it is not a property line, it is a item from a list.

This allows to have a single functions testing properties.


Edit and save the NMREDATA tags

Data-parser-ok.png A preliminary implementation allows to edit the content of NMREDATA tags and save the changes into memory (not yet to a file). The effect of the change may be tested immediately on the molecular structure, in the case of assignments and couplings.

Analyse the individual NMR tag

Depending on the program, what data will be extracted will vary, but all Properties (whether they are understood or not) should be send to the output when writing the file. They don't need to be analysed. Same argument for the properties of peaks. They should all be stored (even if not understood) to be written later.

<NMREDATA_ASSIGNMENT>

Data-parser-ok.png List of assignments is read and displayed on the structure using labels on each atom referenced. Labels for implicit Hydrogens are added to those of the heavy atom, as 2nd, 3rd lines.

Data-parser-ok.png Labels must come enclosed in <""> when they include "," "/" "\" "|" or "&" characters. All these have been tested and are properly working:

a
H-C(1)
Proton_1
H'
H
Ha or Hb
<"Ha,Hb">
<"Ha/Hb">
<"Ha\Hb">
<"Ha|Hb">
<"Ha&Hb">
<"Ha & Hb">
<"Ha,Hb,Hc">
<"Ha/Hb/Hc">
<"Ha\Hb\Hc">
<"Ha|Hb|Hc">
<"Ha&Hb&Hc">
<"Ha & Hb & Hc">

Data-parser.png The possibility that the assignment may be vague. Interchangeable or ambiguous assignment, with the keyword "Interchangeable=".

  • Pre-implemented: it is not displayed, but it is detected and a warning alert is displayed (with a count).

Data-parser.png Equivalent spins, with the keyword "Equivalent=".

  • Pre-implemented: it is not displayed, but it is detected and a warning alert is displayed (with a count).

When there are two structures (e.g. 2D and 3D) in the same SDF file, we might need one ASSIGNMENT tag per structure.

  • If the same atom labels are in both models, maybe just one ASSIGNMENT tag is valid. This needs investigation.
  • They may also be different with respect to the "Interchangeable=" lines.


<NMREDATA_J> (couplings)

Data-parser.png This tag includes atom labels as defined in <NMREDATA_ASSIGNMENT>. Proposal:

  • Display couplings on the structure using a line that connects the two atoms involved.
    • Problem: when there are implicit Hydrogens involved in the coupling, the line may go to the respective heavy atom but needs to be differentiated.
    • Problem: the heavy atom may not have a label in the Assignment tag.
    • Problem: too crowded.
  • Show the value of the J constant as a text label near the line.
  • If present, display the "number of bonds" using different colours for the lines and labels.


<NMREDATA_1D_1H>

Spectrum data, also contains atom labels and couplings.

Data-parser.png

  • If there is content about couplings in the NMREDATA_J tag, should we ignore the coupling information inside the 1D_1H tag?
  • If the NMREDATA_J tag is empty (unassigned couplings),
    • and the coupling information in 1D_1H does not include atom labels, what to display?
    example: 4.1823, S=dd, N=1, L=a J=9.30,4.80;
    • and the coupling information in 1D_1H includes atom labels, try to display the coupled pairs of atoms on the structure.
    example: 4.1823, S=dd, N=1, L=a J=9.30(b), 4.80(c);
  • How to manage the display of couplings when there are several spectrum tags? (e.g. 1D_1H, 1D_13C, 2D_13C_1J_1H)

<NMREDATA_ALATIS> (optional but desired)

Here should come the ALATIS code of the compound. (If possible, it should be included!)

Data-parser.png Pre-implemented: access the ALATIS server, send the 2D SDF moldata and retrieve both the 3D structure, the ALATIS Key code and the InChI. These might be stored by adding them to new tags in the file.


Reading other files in the NMReDATA zipfile

Data-parser-ok.png When clicked on an item in the directory (file tree),

  • If the file is known to be in plain-text format, its content is displayed in the central section.
    Extensions: txt, temp, output, info, par, xml
    Filenames: acqu*, gpnam*, spnam*, proc*, title, pulseprogram, wave
    Specific (reserved for internal use): JmolManifest
  • If the file is known to be binary, its content cannot be displayed; an explanation warning is shown. Same when the file type is not known for sure.
    Filenames: fid, ser, 1r, 1i, 2rr, 2ir, 2ri, 2ii, peaks
  • If the file has PNG extension, it is displayed (image, thumbnail of a spectrum) in the central column (scaled 2x).
  • If the item is a subfolder with numeric name, the thumb.png file that is inside is displayed (may be several levels below). This is currently broken
  • If the item has a DX, JDX or JCAMP extension, its contents (JCAMP spectrum) are displayed in JSpecView within the rightmost column and also as plain text in the central column.


Generation of a 3D molecular structure

Data-parser.png Two methods are currently available for calculating a reasonable 3D structure. They also involve adding the implicit hydrogens which are not included in the original 2D structure.

  • Using JSmol capabilities.
  • Sending the structure to the ALATIS server and retrieving the 3D structure data.

Caveats:

  • Getting the proper stereo configuration is not possible in an automated way. We need a wizard manually assisted and validated by the user. Implementation has started for this but it still needs much work.
  • The added atoms should be labelled in accordance with the data in the ASSIGNMENT tag or spectrum tags.
  • Ensure the addition of all H's in a single operation (*).

(*)The simplest JSmol method fails to add all H's to some flat models. Two addition steps with a 3D optimisation in between are needed.

So we are now testing 2 enhanced methods:

  1. Random push of the C's out of the z=0 plane + add hydrogens + short optimisation + add hydrogens again + full optimisation.
  2. Find C's with 3 bonds to heavy atoms (these atoms seem to be those with *problems in the original simple method) + attach a new hydrogen to them, positioned out of the z=0 plane + run a full optimisation.

Testing is need to decide which method is better.

In both cases, further runs of optimisation may be necessary, driven by the user with the aid of numerical reports of the drop in energy (convergence of he minimisation).

Addition of H's to CR2 is particularly problematic:

  • No problem when the two H are isochronous and the two H's will receive the same label with and without primes. The option should be offered to add "a"/"b" labels, or else just allow edition of the labels.
  • When the two H are not isochronous, user intervention is required to assign them properly or, if the answer is unknown, to use the "Interchangeable=label1, labels\" line in the ASSIGNMENT tag. Possibly four options:
    1. correct
    2. reverse (flip the two labels)
    3. ignore (write no "interchangeable" line)
    4. flag labels as Interchangeable

We could have a link to provide some additional explanation in a wiki page. If this text could be in a separate file, it could be edited easily without changing the "code" part of the page. (Nice for updates - this is not necessary.)

Desirable: to display in a different style the bonds with unknown chirality.

Question: is it necessary to modify the 2D file according to the result of the 3D model and its labeling?