docs updated

README.md

@@ -55,21 +55,34 @@ python -m unittest discover -s tests
 Instructions
 -------------
 
-Before you run, please read the `data/datamodel.rst` file. The instructions about the input and output file are provided there.
+Before running the program, please read the `data/datamodel.rst` file. The instructions for the input and output files are provided there. I have also provided an example QSO spectra FITS file, `data/qso_test.fits`, which contains 500 continuum-normalized SDSS QSO spectra. You can use this file to test an example run as described below.
 
 Running example:
 ----------------
 
 ```sh
-qsoabsfind --input-fits-file path_to_input_fits_file.fits \
-           --n-qso 1-1000:10 \
+qsoabsfind --input-fits-file data/qso_test.fits \
+           --n-qso 500 \
            --absorber MgII \
-           --output path_to_output_fits_file.fits \
-           --headers KEY1=VALUE1 KEY2=VALUE2 \
+           --output test_MgII.fits \
+           --headers SURVEY=SDSS AUTHOR=YOUR_NAME \
            --n-tasks 16 \
            --ncpus 4
 ```
 
+Useful notes:
+-------------
+
+Parallel mode can be memory-intensive if the input FITS file is large in size. As the code accesses the FITS file to read QSO spectra when running in parallel, it can become a bottleneck for memory, and the code may fail. Currently, I suggest the following:
+
+    - **Divide your file into smaller chunks:** Split the FITS file into several smaller files, each containing approximately `N` spectra. Then run the code on these smaller files.
+
+    - **Use a rule of thumb for file size:** Ensure that the size of each individual file is no larger than `total_memory/ncpu` of your node or system. Based on this idea you can decide your `N`. I would suggest `N = 1000`.
+
+    - **Merge results at the end:** After processing, you can merge your results.
+
+In order to decide the right size of the FITS file, consider the total available memory and the number of CPUs in your system.
+
 Contribution
 ------------
 
@@ -93,4 +106,4 @@ Thanks,
 Abhijeet Anand  
 Lawrence Berkeley National Lab  
 
-If you have any questions/suggestions, please feel free to write to [email protected]
+If you have any questions/suggestions, please feel free to write to [email protected] or, preferably, open a GitHub issue..

docs/_build/html/.buildinfo

@@ -1,4 +1,4 @@
 # Sphinx build info version 1
 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: b4c1ffa8d26777bd7d14e5c5ac1f17da
+config: 8b55f0c79b7a490a3b5a9c02c39d73dd
 tags: 645f666f9bcd5a90fca523b33c5a78b7

docs/_build/html/_modules/index.html

@@ -89,7 +89,7 @@ <h1>All modules for which code is available</h1>
 
   <div role="contentinfo">
     <p>&#169; Copyright 2021-2024, Abhijeet Anand.
-      <span class="lastupdated">Last updated on True.
+      <span class="lastupdated">Last updated on 2024-08-01 23:30:26.821787.
       </span></p>
   </div>
 

docs/_build/html/_modules/qsoabsfind/ew.html

@@ -73,11 +73,18 @@
            <div itemprop="articleBody">
 
   <h1>Source code for qsoabsfind.ew</h1><div class="highlight"><pre>
-<span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
+<span></span><span class="sd">&quot;&quot;&quot;</span>
+<span class="sd">This script contains a function to fit a given absorption profile with a double gaussian and measure equivalent widths.</span>
+<span class="sd">&quot;&quot;&quot;</span>
+
+<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
 <span class="kn">from</span> <span class="nn">scipy.optimize</span> <span class="kn">import</span> <span class="n">curve_fit</span>
-<span class="kn">from</span> <span class="nn">.config</span> <span class="kn">import</span> <span class="n">lines</span>
 <span class="kn">from</span> <span class="nn">.utils</span> <span class="kn">import</span> <span class="n">double_gaussian</span>
 <span class="kn">from</span> <span class="nn">.absorberutils</span> <span class="kn">import</span> <span class="n">redshift_estimate</span>
+<span class="kn">from</span> <span class="nn">.config</span> <span class="kn">import</span> <span class="n">load_constants</span>
+
+<span class="n">constants</span> <span class="o">=</span> <span class="n">load_constants</span><span class="p">()</span>
+<span class="n">lines</span> <span class="o">=</span> <span class="n">constants</span><span class="o">.</span><span class="n">lines</span>
 
 <span class="c1"># Example usage within double_curve_fit</span>
 <div class="viewcode-block" id="double_curve_fit">
@@ -350,7 +357,9 @@ <h1>Source code for qsoabsfind.ew</h1><div class="highlight"><pre>
   <hr/>
 
   <div role="contentinfo">
-    <p>&#169; Copyright .</p>
+    <p>&#169; Copyright 2021-2024, Abhijeet Anand.
+      <span class="lastupdated">Last updated on 2024-08-01 23:18:45.166774.
+      </span></p>
   </div>
 
   Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a

docs/_build/html/_modules/qsoabsfind/io.html

@@ -73,40 +73,44 @@
            <div itemprop="articleBody">
 
   <h1>Source code for qsoabsfind.io</h1><div class="highlight"><pre>
-<span></span><span class="kn">import</span> <span class="nn">astropy.io.fits</span> <span class="k">as</span> <span class="nn">fits</span>
+<span></span><span class="sd">&quot;&quot;&quot;</span>
+<span class="sd">This script contains a functions to read and write files.</span>
+<span class="sd">&quot;&quot;&quot;</span>
+
+<span class="kn">import</span> <span class="nn">astropy.io.fits</span> <span class="k">as</span> <span class="nn">fits</span>
 <span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
 
 <div class="viewcode-block" id="read_fits_file">
 <a class="viewcode-back" href="../../qsoabsfind.html#qsoabsfind.io.read_fits_file">[docs]</a>
 <span class="k">def</span> <span class="nf">read_fits_file</span><span class="p">(</span><span class="n">fits_file</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
 <span class="w">    </span><span class="sd">&quot;&quot;&quot;</span>
-<span class="sd">    Reads the FLUX, ERROR, WAVELENGTH, and TGTDETAILS extensions from the</span>
+<span class="sd">    Reads the FLUX, ERROR, WAVELENGTH, and METADATA extensions from the</span>
 <span class="sd">    FITS file.</span>
 
 <span class="sd">    Args:</span>
 <span class="sd">        fits_file (str): Path to the FITS file containing QSO spectra.</span>
 <span class="sd">        index (int, list, or np.ndarray, optional): Index or indices of the rows to load. Default is None.</span>
 
 <span class="sd">    Returns:</span>
-<span class="sd">        tuple: A tuple containing the flux, error, wavelength, and tgtdetails data.</span>
+<span class="sd">        tuple: A tuple containing the flux, error, wavelength, and metadata data.</span>
 <span class="sd">    &quot;&quot;&quot;</span>
     <span class="k">with</span> <span class="n">fits</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="n">fits_file</span><span class="p">,</span> <span class="n">memmap</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> <span class="k">as</span> <span class="n">hdul</span><span class="p">:</span>
         <span class="k">if</span> <span class="n">index</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
             <span class="n">flux</span> <span class="o">=</span> <span class="n">hdul</span><span class="p">[</span><span class="s1">&#39;FLUX&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">data</span>
             <span class="n">error</span> <span class="o">=</span> <span class="n">hdul</span><span class="p">[</span><span class="s1">&#39;ERROR&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">data</span>
             <span class="n">wavelength</span> <span class="o">=</span> <span class="n">hdul</span><span class="p">[</span><span class="s1">&#39;WAVELENGTH&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">data</span>
-            <span class="n">tgtdetails</span> <span class="o">=</span> <span class="n">hdul</span><span class="p">[</span><span class="s1">&#39;TGTDETAILS&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">data</span>
+            <span class="n">metadata</span> <span class="o">=</span> <span class="n">hdul</span><span class="p">[</span><span class="s1">&#39;METADATA&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">data</span>
         <span class="k">else</span><span class="p">:</span>
             <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">index</span><span class="p">,</span> <span class="nb">int</span><span class="p">):</span>
                 <span class="n">flux</span> <span class="o">=</span> <span class="n">hdul</span><span class="p">[</span><span class="s1">&#39;FLUX&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="n">index</span><span class="p">]</span><span class="o">.</span><span class="n">flatten</span><span class="p">()</span>
                 <span class="n">error</span> <span class="o">=</span> <span class="n">hdul</span><span class="p">[</span><span class="s1">&#39;ERROR&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="n">index</span><span class="p">]</span><span class="o">.</span><span class="n">flatten</span><span class="p">()</span>
-                <span class="n">tgtdetails</span> <span class="o">=</span> <span class="p">{</span><span class="n">name</span><span class="p">:</span> <span class="n">hdul</span><span class="p">[</span><span class="s1">&#39;TGTDETAILS&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="n">index</span><span class="p">][</span><span class="n">name</span><span class="p">]</span> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">hdul</span><span class="p">[</span><span class="s1">&#39;TGTDETAILS&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">names</span><span class="p">}</span>
+                <span class="n">metadata</span> <span class="o">=</span> <span class="p">{</span><span class="n">name</span><span class="p">:</span> <span class="n">hdul</span><span class="p">[</span><span class="s1">&#39;METADATA&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="n">index</span><span class="p">][</span><span class="n">name</span><span class="p">]</span> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">hdul</span><span class="p">[</span><span class="s1">&#39;METADATA&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">names</span><span class="p">}</span>
             <span class="k">else</span><span class="p">:</span>
                 <span class="n">flux</span> <span class="o">=</span> <span class="n">hdul</span><span class="p">[</span><span class="s1">&#39;FLUX&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="n">index</span><span class="p">]</span>
                 <span class="n">error</span> <span class="o">=</span> <span class="n">hdul</span><span class="p">[</span><span class="s1">&#39;ERROR&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="n">index</span><span class="p">]</span>
-                <span class="n">tgtdetails</span> <span class="o">=</span> <span class="n">hdul</span><span class="p">[</span><span class="s1">&#39;TGTDETAILS&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="n">index</span><span class="p">]</span>
+                <span class="n">metadata</span> <span class="o">=</span> <span class="n">hdul</span><span class="p">[</span><span class="s1">&#39;METADATA&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="n">index</span><span class="p">]</span>
             <span class="n">wavelength</span> <span class="o">=</span> <span class="n">hdul</span><span class="p">[</span><span class="s1">&#39;WAVELENGTH&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">data</span>  <span class="c1"># Assuming wavelength is common for all spectra</span>
-    <span class="k">return</span> <span class="n">flux</span><span class="p">,</span> <span class="n">error</span><span class="p">,</span> <span class="n">wavelength</span><span class="p">,</span> <span class="n">tgtdetails</span></div>
+    <span class="k">return</span> <span class="n">flux</span><span class="p">,</span> <span class="n">error</span><span class="p">,</span> <span class="n">wavelength</span><span class="p">,</span> <span class="n">metadata</span></div>
 
 
 <div class="viewcode-block" id="save_results_to_fits">
@@ -167,7 +171,9 @@ <h1>Source code for qsoabsfind.io</h1><div class="highlight"><pre>
   <hr/>
 
   <div role="contentinfo">
-    <p>&#169; Copyright .</p>
+    <p>&#169; Copyright 2021-2024, Abhijeet Anand.
+      <span class="lastupdated">Last updated on 2024-08-01 23:18:45.166774.
+      </span></p>
   </div>
 
   Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a