17.4: Datasets

In pandas, a DataFrame can be thought of as a collection of Series objects that share a common index, allowing for the alignment of data across rows. This structure is exceedingly efficient for handling one-dimensional sequences of values, typically time series or other forms of sequentially indexed data, which are common in various domains including finance, economics, and simple observational studies.

Similarly, Xarray Datasets can be thought of as a collection of DataArrays with common coordinates, similar to the way a Pandas DataFrame is a collection of Series. Xarray's Dataset enables the alignment and joint operation of multiple DataArrays across their shared coordinates, providing a robust framework for managing the intricacies of multidimensional datasets, like those generated from climate models or satellite observations.

Here is one example of how you would create a Dataset:

import numpy as np
import pandas as pd
import xarray as xr

# Define the coordinates
latitudes = np.linspace(-90, 90, 181)   # 181 points from South to North Pole
longitudes = np.linspace(-180, 180, 361) # 361 points for full longitude range
pressure_levels = np.array([1000, 850, 700, 500, 300, 200, 100])  # in hPa
times = pd.date_range('2005-01-01', '2005-12-01', freq='MS')  # Monthly intervals for 2005

# Generate random data
temperature_data = np.random.rand(len(times), len(pressure_levels), len(latitudes), len(longitudes))
precipitation_data = np.random.rand(len(times), len(pressure_levels), len(latitudes), len(longitudes))

# Create the xarray Dataset
ds = xr.Dataset({
    'temperature': (['time', 'pressure', 'latitude', 'longitude'], temperature_data),
    'precipitation': (['time', 'pressure', 'latitude', 'longitude'], precipitation_data)
}, coords={
    'time': times,
    'pressure': pressure_levels,
    'latitude': latitudes,
    'longitude': longitudes
})

In this example, the xr.Dataset constructor is used to create a new Dataset. The first argument is a dictionary that maps variable names to dimension names and data arrays.  The coords argument is a dictionary that assigns coordinate labels to the dimensions. The times array generated with pd.date_range provides monthly intervals throughout the year 2005, while latitudes and longitudes are evenly spaced points on the globe, and pressure_levels represent typical atmospheric pressure levels in hPa.

Accessing the variables within an Xarray dataset is straightforward, similar to how one might interact with a dictionary in Python. Each variable in the dataset is a Xarray DataArray, which can be retrieved using the variable name as the key. For example, to access the 'temperature' variable from the dataset `ds` created in the previous example, you simply use `ds['temperature']` or `ds.temperature`. This retrieves the DataArray corresponding to temperature, complete with its associated dimensions, coordinates, and attributes. This structure allows for intuitive querying and manipulation of the data. You can similarly access the 'precipitation' DataArray using `ds['precipitation']` or `ds.precipitation`. These variables can be handled independently or in conjunction with other dataset variables, depending on the analysis required. It's this simplicity in variable access that makes Xarray a potent tool for handling high-dimensional datasets typical in fields like atmospheric sciences.

When performing calculations or operations on an Xarray dataset, the operation propagates across all the included variables, applying the computation en masse. This attribute of Xarray is particularly advantageous for dataset-wide adjustments or analyses. Consider a scenario where we want to add one to every data point in the dataset. We could do this with the command `ds = ds + 1`, which would add one to each variable in `ds`.

Similarly, when performing coordinate-based selections, the operation is universally applied. If you were to select a subset of data along a specific coordinate, say a particular range of latitudes, executing something like `ds.sel(latitude=slice(10, 20))` would concurrently extract the slice for all variables within `ds` that correspond to those latitudes. It would return a new dataset with the same variables but with the requested range of latitudes.  Remember that this is a view, not a copy, so you may get irregular results if you try to modify the contents of the view.