brimfile

  1"""
  2## What is brimfile?
  3
  4*brimfile* is a Python library to read from and write to brim (**Br**illouin **im**aging) files,
  5which contain both the spectra and analysed data for Brillouin imaging.
  6More information about the brim file format can be found [here](https://github.com/prevedel-lab/Brillouin-standard-file).
  7
  8Briefly, a brim file can contain multiple data groups,
  9typically corresponding to imaging of the same sample at different timepoints/conditions.
 10Each data group contains the spectral data as well as the metadata and
 11the results of the analysis on the spectral data (which can be many in case multiple reconstruction pipelines are performed).
 12
 13The structure of the *brimfile* library reflects the structure of the brim file and the user can access the data,
 14metadata and analysis results through their corresponding classes.
 15
 16- [File](#file): represents a brim file, which can be opened or created.
 17- [Data](#data): represents a data group in the brim file, which contains the spectral data and metadata.
 18- [Metadata](#metadata): represents the metadata associated to a data group (or to the whole file).
 19- [AnalysisResults](#analysisresults): represents the results of the analysis of the spectral data.
 20
 21
 22## Install brimfile
 23
 24We recommend installing *brimfile* in a [virtual environment](https://docs.python.org/3/library/venv.html).
 25
 26After activating the new environment, simply run:
 27
 28```bash
 29pip install brimfile
 30```
 31
 32If you also need the support for exporting the analyzed data to OME-TIFF files,
 33you can install the optional dependencies with:
 34
 35```bash
 36pip install "brimfile[export-tiff]"
 37```
 38
 39For accessing remote data (i.e. S3 buckets), you need `remote-store`:
 40
 41```bash
 42pip install "brimfile[remote-store]"
 43```
 44
 45## Quickstart
 46
 47The following code shows how to:
 48- open a .brim file 
 49- get an image for the Brillouin shift 
 50- get the spectrum at a specific pixel
 51- get the metadata.
 52
 53```Python
 54from brimfile import File, Data, Metadata, AnalysisResults
 55Quantity = AnalysisResults.Quantity
 56PeakType = AnalysisResults.PeakType
 57
 58filename = 'path/to/your/file.brim.zarr' 
 59f = File(filename)
 60
 61# get the first data group in the file
 62d = f.get_data()
 63
 64# get the first analysis results in the data group
 65ar = d.get_analysis_results()
 66
 67# get the image for the shift
 68img, px_size = ar.get_image(Quantity.Shift, PeakType.average)
 69
 70# get the spectrum at the pixel (pz,py,px)
 71(pz,py,px) = (0,0,0)
 72PSD, frequency, PSD_units, frequency_units = d.get_spectrum_in_image((pz,py,px))
 73
 74# get the metadata 
 75md = d.get_metadata()
 76all_metadata = md.all_to_dict()
 77
 78# close the file
 79f.close()
 80```
 81
 82## Store types
 83
 84Currently brimfile supports zip, zarr and S3 buckets as a store.
 85When opening or creating a file, the storage be selected by using the brimfile.file_abstraction.StoreType enum; zip and zarr can be used both for reading and writing while S3 only for reading. 
 86
 87Although it is possible to write directly to zip, this will create duplicated entries in the archive (see [GitHub issue](https://github.com/zarr-developers/zarr-python/issues/1695)).
 88
 89A possible workaround is to create a .zarr store instead and zip the folder afterwards.
 90Importantly the root of the archive should not contain the folder itself, i.e. you should go inside the .zarr folder, select all the elements there, right click on them to create a .zip archive.
 91
 92
 93## Use brimfile
 94
 95### File
 96
 97The main class is `brimfile.file.File`, which represents a brim file.
 98It can be used to create a new brim file (`brimfile.file.File.create`) or to open an existing one (`brimfile.file.File.__init__`).
 99
100```Python
101import brimfile as brim
102
103filename = 'path/to/your/file.brim.zarr'
104
105# Open an existing brim file
106f = brim.File(filename)
107
108# or create a new one
109f = brim.File.create(filename)
110```
111
112### Data
113
114You can then get a `brimfile.data.Data` object representing the data group in the brim file
115by opening an existing one (`brimfile.file.File.get_data`).
116
117```Python
118# Get the first data group in the file
119data = f.get_data()
120```
121
122To add a new data group to the file, you can use the `brimfile.file.File.create_data_group` method,
123which accepts a 4D array for the PSD with dimensions (z, y, x, spectrum),
124a frequency array which might have the same size as PSD or be 1D, in case the frequency axis is the same for all the spectra.
125```Python
126# or create a new one
127data = f.create_data_group(PSD, freq_GHz, (dz, dy, dx), name='my_data_group')
128```
129Alternatively you can use `brimfile.file.File.create_data_group_sparse` for sparse data, which lets you directly assign the correspondence
130between the spatial positions and the spectra through the `scanning` dictionary.
131
132Once you have an istance of `brimfile.data.Data`, you can get the spectrum corresponding to a pixel in the image
133by calling the `brimfile.data.Data.get_spectrum_in_image` method:
134```Python
135PSD, frequency, PSD_units, frequency_units = data.get_spectrum_in_image((pz,py,px))    
136```
137
138### Metadata
139
140You can then get a `brimfile.metadata.Metadata` object by simply calling the `brimfile.data.Data.get_metadata` method on a previously retrieved `Data` object.
141The returned Metadata object contains all the metadata associated with the file and the data group.
142```Python
143metadata = data.get_metadata()
144```
145The list of available metadata is defined [here](https://github.com/prevedel-lab/Brillouin-standard-file/blob/main/docs/brim_file_metadata.md).
146
147New metadata can be added to the current data group (or to the whole file) by calling the `brimfile.metadata.Metadata.add` method.
148```Python
149import datetime
150
151Attr = Metadata.Item
152datetime_now = datetime.now().isoformat()
153temp = Attr(22.0, 'C')
154    
155metadata.add(Metadata.Type.Experiment, {'Datetime':datetime_now, 'Temperature':temp},local=True)
156```
157A single metadata item can be retrieved by indexing the `Metadata` object, which takes a string in the format 'group.object', e.g. 'Experiment.Datetime'.
158```Python
159datetime = metadata['Experiment.Datetime']
160```
161A dictionary containing all metadata can be retrieved by calling the `brimfile.metadata.Metadata.all_to_dict` method.
162```Python
163metadata.all_to_dict()
164```
165
166### AnalysisResults
167
168The results of the analysis can be accessed through the `brimfile.analysis_results.AnalysisResults` object, obtained by calling the `brimfile.data.Data.get_analysis_results` method on a previously retrieved `Data` object:
169``` Python
170analysis_results = data.get_analysis_results()
171```
172or create a new one by calling the `brimfile.data.Data.create_analysis_results_group`:
173``` Python
174analysis_results = data.create_analysis_results_group(shift, width,
175    name='my_analysis_results')
176```
177
178`AnalysisResults` also exposes a method to retrieve the images of the analysis results (`brimfile.analysis_results.AnalysisResults.get_image`):
179
180``` Python
181ar_cls = AnalysisResults
182img, px_size = analysis_results.get_image(ar_cls.Quantity.Shift, ar_cls.PeakType.average)
183```
184
185## List the contents of a brim file
186
187The *brimfile* library provides methods to list the contents of a brim file.
188
189To list all the data groups in a brim file, you can use the `brimfile.file.File.list_data_groups` method.
190
191Once you have a `Data` object, you can list the analysis results in it by calling the `brimfile.data.Data.list_AnalysisResults` method.
192
193Once you have an `AnalysisResults` object, you can determine:
194- if the Stokes and/or anti-Stokes peaks are present by calling the `brimfile.analysis_results.AnalysisResults.list_existing_peak_types` method;
195- the available quantities (e.g. shift, linewidth, etc...) in the analysis results by calling the `brimfile.analysis_results.AnalysisResults.list_existing_quantities` method.
196
197## Example code
198
199Here is a simple example which creates a brim file with a data group and some metadata and then reads it back.
200
201We first write a function to generate some dummy data:
202
203``` Python
204import numpy as np
205
206def generate_data():
207    def lorentzian(x, x0, w):
208        return 1/(1+((x-x0)/(w/2))**2)
209    Nx, Ny, Nz = (7, 5, 3) # Number of points in x,y,z
210    dx, dy, dz = (0.4, 0.5, 2) # Stepsizes (in um)
211    n_points = Nx*Ny*Nz  # total number of points
212
213    width_GHz = 0.4
214    width_GHz_arr = np.full((Nz, Ny, Nx), width_GHz)
215    shift_GHz_arr = np.empty((Nz, Ny, Nx))
216    freq_GHz = np.linspace(6, 9, 151)  # 151 frequency points
217    PSD = np.empty((Nz, Ny, Nx, len(freq_GHz)))
218    for i in range(Nz):
219        for j in range(Ny):
220            for k in range(Nx):
221                index = k + Nx*j + Ny*Nx*i
222                #let's increase the shift linearly to have a readout 
223                shift_GHz = freq_GHz[0] + (freq_GHz[-1]-freq_GHz[0]) * index/n_points
224                spectrum = lorentzian(freq_GHz, shift_GHz, width_GHz)
225                shift_GHz_arr[i,j,k] = shift_GHz 
226                PSD[i, j, k,:] = spectrum
227
228    return PSD, freq_GHz, (dz,dy,dx), shift_GHz_arr, width_GHz_arr
229```
230
231Now we can use this function to create a brim file with a data group and some metadata:
232
233``` Python
234    from brimfile import File, Data, Metadata, StoreType
235    from datetime import datetime
236
237    filename = 'path/to/your/file.brim.zarr' 
238
239    f = File.create(filename, store_type=StoreType.AUTO)
240
241    PSD, freq_GHz, (dz,dy,dx), shift_GHz, width_GHz = generate_data()
242    
243    d0 = f.create_data_group(PSD, freq_GHz, (dz,dy,dx), name='test1')
244    
245    # Create the metadata
246    Attr = Metadata.Item
247    datetime_now = datetime.now().isoformat()
248    temp = Attr(22.0, 'C')
249    md = d0.get_metadata()
250    
251    md.add(Metadata.Type.Experiment, {'Datetime':datetime_now, 'Temperature':temp})
252    md.add(Metadata.Type.Optics, {'Wavelength':Attr(660, 'nm')})
253    # Add some metadata to the local data group   
254    temp = Attr(37.0, 'C')
255    md.add(Metadata.Type.Experiment, {'Temperature':temp}, local=True)
256
257    # create the analysis results
258    ar = d0.create_analysis_results_group({'shift':shift_GHz, 'shift_units': 'GHz',
259                                             'width': width_GHz, 'width_units': 'Hz'},
260                                             {'shift':shift_GHz, 'shift_units': 'GHz',
261                                             'width': width_GHz, 'width_units': 'Hz'},
262                                             name = 'test1_analysis')
263    f.close()
264```
265and we can read it back:
266``` Python
267    from brimfile import File, Data, Metadata, AnalysisResults
268
269    filename = 'path/to/your/file.brim.zarr' 
270
271    f = File(filename)
272
273    # check if the file is read only
274    f.is_read_only()
275
276    #list all the data groups in the file
277    data_groups = f.list_data_groups(retrieve_custom_name=True)
278
279    # get the first data group in the file
280    d = f.get_data()
281    # get the name of the data group
282    d.get_name()
283
284    # get the number of parameters which the spectra depend on
285    n_pars = d.get_num_parameters()
286
287    # get the metadata 
288    md = d.get_metadata()
289    all_metadata = md.all_to_dict()
290    # the list of metadata is defined here https://github.com/prevedel-lab/Brillouin-standard-file/blob/main/docs/brim_file_metadata.md
291    time = md['Experiment.Datetime']
292    time.value
293    time.units
294    temp = md['Experiment.Temperature']
295    md_dict = md.to_dict(Metadata.Type.Experiment)
296
297
298    #get the list of analysis results in the data group
299    ar_list = d.list_AnalysisResults(retrieve_custom_name=True)
300    # get the first analysis results in the data group
301    ar = d.get_analysis_results()
302    # get the name of the analysis results
303    ar.get_name()
304    # list the existing peak types and quantities in the analysis results
305    pt = ar.list_existing_peak_types()
306    qt = ar.list_existing_quantities()
307    # get the image of the shift quantity for the average of the Stokes and anti-Stokes peaks
308    img, px_size = ar.get_image(AnalysisResults.Quantity.Shift, AnalysisResults.PeakType.average)
309    # get the units of the shift quantity
310    u = ar.get_units(AnalysisResults.Quantity.Shift)
311
312    # get a quantity at a specific pixel (coord) in the image
313    coord = (1,3,4)
314    qt_at_px = ar.get_quantity_at_pixel(coord, AnalysisResults.Quantity.Shift, AnalysisResults.PeakType.average)
315    assert img[coord]==qt_at_px
316    
317    # get the spectrum in the image at a specific pixel (coord)
318    PSD, frequency, PSD_units, frequency_units = d.get_spectrum_in_image(coord)    
319
320    f.close()
321```
322
323## Export the data to a different format
324
325### OME-TIFF
326
327You can export a specific quantity in the analyzed data to OME-TIFF files using the `brimfile.analysis_results.AnalysisResults.save_image_to_OMETiff` method on an instance `ar` of the `AnalysisResults` class.
328``` Python
329ar_cls = AnalysisResults
330ar.save_image_to_OMETiff(ar_cls.Quantity.Shift, ar_cls.PeakType.average, filename='path/to/your/exported_tiff' )
331```
332"""
333
334__version__ = "1.4.2"
335
336from .file import File
337from .data import Data
338from .analysis_results import AnalysisResults
339from .metadata import Metadata
340from .file_abstraction import StoreType