Skip to content

Latest commit

 

History

History
630 lines (495 loc) · 31.2 KB

2024.md

File metadata and controls

630 lines (495 loc) · 31.2 KB
Raw

Fatiando Development Calls 2024

📱 Join the video call: https://meet.jit.si/fatiando-a-terra

📅 Shared calendar: https://www.fatiando.org/calendar

IMPORTANT: Everyone is required to follow our Code of Conduct when participating in the Fatiando community. Please review it carefully.

Development calls are open to everyone! Here we discuss project development, socialize, and do some live coding from time to time.

Everyone is encouraged to participate and edit the notes below.

2024-05-14

Time: 14:00 UTC

Participants:

  • Santi
  • Mariana
  • Leo

Quick updates:

  • Next meeting we'll have Lu's talk about his adventures in Antarctica 🐧!
    • We should promote it on Mattermost, social media, etc
    • Live streaming? Or just record it and then upload?
    • Ask Lu if we can live stream it
    • We can live stream it directly from Jitsi
      • Make a practice run before the talk
      • Need Youtube API and start the stream on Youtube!
    • We can schedule the live stream ahead of time. This way we can share the link in social media, add it to the calendar, etc
  • Santi's maintenance works:
    • Replace Sphinx napoleon for numpydoc in multiple libraries: #149
      • These changes would improve how we show the API docs of classes (solving issues with attributes that have a trailing underscode, like coefs_).
      • Check if we can also list methods as a toc on the right menu.
    • Solved issues with tests failing in MacOS runners using the oldest dependencies: fatiando/harmonica#499
      • New macos GitHub runners make use of Apple silicon (M1, M2 chips). Numpy for Python 3.8 is not available for those chips.
      • Solved the issue by running the Python 3.8 tests on the latest x86 (intel chips) macos runners.
    • Codecov failing when trying to upload multiple files (specially when PR is opened from a fork)
      • Working on solving it by uploading coverage reports as artifacts and uploading them all on a single run
      • fatiando/harmonica#502
  • Leo's nerdy week:
    • Associated Legendre functions! Did a deep dive and started implementing (almost finished, just speeding them up now) fatiando/harmonica#505
    • IGRF in Harmonica: fatiando/harmonica#504
    • It's fine to add Boule as a dependency
class IGRF(date, ellipsoid=boule.WGS84):
    ...

IGRF(datetime(1990, 3, 14)).predict((lon, lat, height), field="b")
IGRF(datetime(1990, 3, 14)).grid(grid_coords, field="b")

sh = MagneticSphericalHarmonics(g, h, ref_radius=..., normalization="schmidt", ellipsoid=None)
sh.predict(..., field="b")
sh.grid(..., field="b")

sh = MagneticSphericalHarmonics(max_degree, ref_radius=..., normalization="schmidt", ellipsoid=None)
sh.fit(
    be=((lon, lat, height), be), 
    bn=((lon, lat, height), bn), 
    bu=((lon, lat, height), bu), 
)

sh = GravitySphericalHarmonics(max_degree, ref_radius=..., normalization="full", ellipsoid=None)
sh.fit(
    gz=((lon, lat, height), gz), 
    gxx=((lon, lat, height), gxx), 
)
sh.predict((lon, lat, height), field="gz")

2024-05-07

Time: 14:00 UTC

Participants:

  • India
  • Santi

Quick updates:

  • Harmonica CI's failing on MacOS with Python 3.8
    • Not being able to install Numpy with pip
    • Trying to fix it: #496
  • Pair programming on India's PR for changing default window_size in gradient-boosted eqs:
    • fatiando/harmonica#487
    • If the passed coordinates have less than 5e3 points, then a warning is raised and a single window will be used.
    • The self.window_size_ is set to None in that case, since it's not being used for defining the window.
    • Few ideas to improve this: use slice instead of np.arange in the intermediate returns.

2024-04-30

Time: 17:00 UTC

Participants:

  • Mariana
  • Santi (only during the end)

Quick updates:

  • Santi talk is available in Zoom, can I download it and publishing in Fatiando Youtube Channel? Slides and notebooks: https://github.com/santisoler/2024-uh
    • Yes! I got permission to publish it. Maybe check if there are bits that should be removed?
  • Today's is Fatiando birthday. Santi check the git log to understand better the transition. Before April 30th 2010, Fatiando was written in cpp (Pregnancy stage)
    • Post on LinkedIn ✔️
    • Post on Mattermost ✔️
    • Post on Mastodon
    • Post on Twitter

2024-04-23

Time: 14:00 UTC

Participants:

  • Lu
  • Leo
  • India
  • Santi
  • Mariana

Quick updates:

2024-04-16

Time: 17:00 UTC

Participants:

  • Santi
  • Leo Miquelutti
  • Mariana
  • India
  • Martin

Quick updates:

  • Codedov push failing in fatiando/harmonica#486
    • Reran the test to see if it pass nows
  • 🚀🎉 Choclo v0.2.0 has been released! 🎉🚀
  • Roadmap to Harmonica v0.7.0:
  • Martin
    • Progress on the xml reader grd files
    • Able to read the wellknown_epsg attribute on the projection tag
    • Add a private function to read the xml file
    • We talked about the signature of that function: it's better if it returns a dictionary witht the "EPSG" value inside. Then the attrs dictionary can be extended before passing it to the xarray object.
      • This would allow us to extend the reader of xml without having to change other code.
  • Chat on #487
    • We need to add tests checking if the window size is set properly
    • Maybe defining a grid with N times 5_000 points and comparing the value of window size it gets assigned against the expected one. We should use numpy.testing.assert_allclose() for that since we are comparing floats.
  • Plan for Lu's talk on one of the 14:00 GMT meetings! 🐧

2024-04-09

Time: 14:00 UTC

Participants:

  • Lu
  • Mariana
  • Santi
  • Leo

Quick updates:

2024-04-02

Time: 17:00 UTC

Participants:

  • Santi
  • India
  • Mariana
  • Leo
  • Martin

Quick updates:

  • Update meeting times
    • Go back to the two times (one every two weeks)
    • 14:00 GMT sounds like the best option
  • Discussions around arguments for equivalent sources:
  • India's workflow:
    • JupyterHub (because here she has configured ssh key with Github) → terminal → git clone repo → mamba create env → conda activate harmonica → git grep EquivalentSourcesGB (to search for which file needs to change) → edit file gradient_boosted.py →
    • To get a nice bash prompt with little effort: https://starship.rs/
  • Mariana's idea: use breakout rooms for tackling multiple issues at the same time

Todo:

  • Update meeting times:
    • Update fatiando calendar (santi)
    • Post on Mattermost (santi)
    • Post on forum (mariana)
    • Post on linkedin (mariana)

2024-03-26

Time: 17:00 UTC

Participants:

  • Mariana
  • India
  • Lu
  • Santi

Quick updates:

  • Question regarding Zenodo citations: fatiando/community@145
  • Done some maintenance work on Choclo and Harmonica
  • I think Leo Miquellutti's PR on total gradient amplitude might be ready to be merged: fatiando/harmonica#478 🎉
  • Lu came back from Antarctica! 🐧
    • Ice drilling and sampling every 5 cm (on 200m hole!)
    • Grav, mag, MT surveys
    • He's willing to give a presentation about his work there
  • We should have another meeting time that's more suitable for Lu
  • Discussion around some Harmonica issues regarding equivalent sources
    • Better default for the depth: we could make it clever enought to set the depth argument as 4.5 times the average distance between the data points. Shall we still allow users to set the depth manually? Should we include a depth_factor argument that defaults to 4.5? Should depth=None by default and in that case the class should compute it with that formula?
    • Better default for window size: set it allowing to have in average 5k data points per window. Maybe we should include the 5_000 as an argument of the class instead of hard coding it.

2024-03-19

Time: 17:00 UTC

Participants:

  • Mariana
  • India
  • Santi
  • Leo

Quick updates:

  • Mariana. New post in LinkedIn promoting community meetings

    WARNING: No ICDs were found. Either, Install a conda package providing a OpenCL implementation (pocl, oclgrind, intel-compute-runtime, beignet) or Make your system-wide implementation visible by installing ocl-icd-system conda package.

🚨 Group programming 🚨

  • We'll continue our work on Harmonica ssue #423
  • Any volunteers to pick up where Santi left off?
  • For Mariana to pull changes from Santi's fork:
    • git remote add santisoler [email protected]:santisoler/harmonica.git
    • git fetch santisoler
    • git switch remove-depth-type
    • git grep depth_type

2024-03-12

Time: 17:00 UTC

Participants:

  • Santiago
  • Mariana
  • Martin
  • India
  • Leo

Quick updates:

  • Maintenance:
    • New tool: Burocrata 🚀
      • Automatically add the copyright notice on every source file
      • The old license_notice.py with a CLI and available in PyPI
    • Using Dependabot for automatically updating the GitHub Actions: fatiando/harmonica#455 (not only in harmonica, but everywhere else as well)
    • Replace PyPI API tokens with Trusted Publisher configuration: #141
  • Started building out Magali: https://www.fatiando.org/magali/dev/overview.html
  • Martin is planning on taking on the Euler Deconvolution issue
  • New filter that we could add: Calculate anomalous field vector B from the total field anomaly. This can be done with FFT. Sort of based on this paper https://doi.org/10.1029/2008JB006006
    • Leo will open an issue
  • Mariana
    • Update svg figure on Inkscape :). Approved by Leo fatiando/tutorials#14
    • Question. Ideas to fix my communist gravity figure?
      • Minor thing: pass projection=projection to eqs.grid
    • Gave a talk at the Uni in Bristol about her PhD and at the end invited people to Fatiando
      • We should add it to the Fatiando news in the website!
  • Santi submitted a talk about Pooch to Scipy. Waiting to see if they'll accept.

2024-03-05

Time: 17:00 UTC

Participants:

  • Mariana
  • India
  • Gelson
  • Santi
  • Leo
  • Agustina

Quick announcements

  • 📣 We are changing time of the meetings to every Tuesday 17:00 GMT

Agenda

🚨 Group programming 🚨

  • The goal is to tackle one of the Harmonica Issues (#423 was chosen last week) in group so we can all learn from the process and the train of thought involved in contributing to Fatiando.
  • One could take the lead and type, while the rest chime in with ideas and questions.
  • The idea is not to make it a live stream where the protagonist is the typer, but to make it as interactive as possible. Other people can take the lead and start typing themselves.
  • Ultimate goals:
    • Allow new contributors to get more familiar with the process
    • Learn about best practices for software development
    • Learn about why we make certain decisions (backward compatibility, raise warnings on breaking changes, adding tests and documentation, etc)
    • Encourage new contributors to open their own PRs

What we did

  • Started working on solving #423
  • Opened a PR: 468
  • We still need to apply changes to gradient boosted equivalent sources

2024-02-29

Time: 17:00 UTC

Participants:

  • Mariana
  • Santi
  • India

Quick updates:

  • Mariana: Working in issue fatiando/tutorials#12

    • How can I get rid of this warning? It means something needs to be changed in Verde? image
    /home/m/anaconda3/envs/fatiando-tutorials/lib/python3.11/site-packages/verde/blockreduce.py:177: FutureWarning: Series.ravel is deprecated. The underlying array is already 1D, so ravel is not necessary.  Use `to_numpy()` for conversion to a numpy array instead.
  columns = {"data{}".format(i): comp.ravel() for i, comp in enumerate(data)}
/home/m/anaconda3/envs/fatiando-tutorials/lib/python3.11/site-packages/verde/blockreduce.py:179: FutureWarning: The provided callable <function mean at 0x7f47517da8e0> is currently using DataFrameGroupBy.mean. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "mean" instead.
  blocked = pd.DataFrame(columns).groupby("block").aggregate(reduction)
/home/m/anaconda3/envs/fatiando-tutorials/lib/python3.11/site-packages/verde/blockreduce.py:236: FutureWarning: The provided callable <function mean at 0x7f47517da8e0> is currently using DataFrameGroupBy.mean. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "mean" instead.
  grouped = table.groupby("block").aggregate(self.reduction)
/home/m/anaconda3/envs/fatiando-tutorials/lib/python3.11/site-packages/verde/blockreduce.py:177: FutureWarning: Series.ravel is deprecated. The underlying array is already 1D, so ravel is not necessary.  Use `to_numpy()` for conversion to a numpy array instead.
  columns = {"data{}".format(i): comp.ravel() for i, comp in enumerate(data)}
/home/m/anaconda3/envs/fatiando-tutorials/lib/python3.11/site-packages/verde/blockreduce.py:179: FutureWarning: The provided callable <function mean at 0x7f47517da8e0> is currently using DataFrameGroupBy.mean. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "mean" instead.
  blocked = pd.DataFrame(columns).groupby("block").aggregate(reduction)
/home/m/anaconda3/envs/fatiando-tutorials/lib/python3.11/site-packages/verde/blockreduce.py:236: FutureWarning: The provided callable <function mean at 0x7f47517da8e0> is currently using DataFrameGroupBy.mean. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "mean" instead.
  grouped = table.groupby("block").aggregate(self.reduction)

    • Santi: these warnings are comming from Pandas (Mariana was using pandas==2.2.1 here). We should check if these are reproduced in Verde and fix them upstream. In summary: it's great the tutorials are raising these warnings, but we should not tackle them in the tutorials repo.

    • For this warning:

      NumbaWarning: The TBB threading layer requires TBB version 2021 update 6 or later i.e., TBB_INTERFACE_VERSION >= 12060. Found TBB_INTERFACE_VERSION = 12050. The TBB threading layer is disabled.
warnings.warn(problem)

  • Changing meeting times:

    • Currently the only ones that can make it to the meeting of 00:00 GMT are the ones in PT (Pacific Time). Until Lu returns to his routine, we could move it to a time that's better for everyone else.
    • Leo cannot attend to the Thursdays 17:00 GMT meetings from now until August. We could move these meetings to same time on Mon, Tue, or Fri.
    • Santi prefers Tue or Fri. One Tue per month it will clash with Swung meeting.
    • TODO:
      • Move all meetings to 17:00 GMT
      • Decide which day might be better for everyone: Tuesday
      • Update Google calendar
      • Write a message in Mattermost letting people know. Let ppl know we are planning to do group programming
      • Post in the forum
      • Check if we need to update the time in fatiando/community

  • Harmonica issues to tackle in the near future:

    • We can go and change the default value of window_size directly without going through the warning-release process: fatiando/harmonica#425
    • For #424 we would make one PR for raising the warnings, and then another one for actually making the change (that will be merged for 0.8)

  • Ideas for future meetings:

    • Do some pair programming or demos/intros on how to code and contribute
    • We can start with #423 next week!

2024-02-15

Time: 17:00 UTC

Participants:

  • Mariana
  • Leo
  • India

Quick updates:

2024-02-01

Time: 17:00 UTC

Participants:

  • Leo
  • Mariana
  • India

Quick updates:

  • Leo: Found a bug in the vertical derivative when implementing Euler deconvolution for a class: fatiando/harmonica#454 Good issue for beginners
  • Leo: Trying to update versions of packages in Verde but running into a few issues. I think it's something on my computer with GMT and sphinx. I keep getting crashes that don't happen on CI. Will try changing OS and see if it keeps happening.
  • India: Do one of the calls as a demo of porting a function or notebook code from a paper/research repo into a package.

2024-01-26

Time: 00:00 UTC

Participants:

  • Santi :-(

Quick updates:

  • Santi won't attend to the next few meetings. See you all back after Feb 15.
  • Harmonica

2024-01-18

Time: 17:00 UTC

Participants:

  • Santi
  • India
  • Mariana
  • Leo
  • Agustina

Quick updates:

  • Merged the forward functions for magnetic prisms: fatiando/harmonica#448
    • Also changed the expected shape of the magnetization vector to better integrate with hm.magnetic_angles_to_vec
    • Plan to start working on the same changes for the forward fucntions for dipoles
  • Merged the update for the forward of point sources using Choclo: fatiando/harmonica#422
    • Thanks to Lu for the work!
    • It might be the first Fatiando PR merged from Antarctica! 🐧
      • Would make a nice post on social media!
  • Opened issue to improve error message while reading a corrupted Oasis Montaj grd file: fatiando/harmonica#451
  • Example running Fatiando and PyGMT on Colab with conda: https://colab.research.google.com/drive/1rfXOt4NSgmnU5XknLoq0nQNqM5j0FzY7?usp=sharing
  • New package: Magali! For processing and inversion of magnetic microscopy.
    • Currently on my lab org (https://github.com/compgeolab/magali) but want to move to Fatiando. Objections? We talked about this before but just confirming.
    • MSc student Yago will arrive in São Paulo in the next month to work on this as his dissertation.
    • Will generate PRs in Harmonica as well (like Euler Deconvolution).
  • New article citing Fatiando: https://jurnal.stageofsleman.id/index.php/jsgs/article/download/11/12
    • Mariana sent an invitation to the corresponding author to join us in the community calls
  • Promoting Dominic Kempf to Developer on Pooch:
    • Has been very active and enthusiastic.
    • We need more hands in the project.
    • Leo will email and CC council to invite.
  • Add MAINTAINERS.md file to repos listing maintainers
    • That file should live in the repo for each package
    • Leo will update the GitHub teams
  • India shared and idea for generating regular grids at angles, useful for "diagonal" flight lines
    • It would be nice to see an issue with more details about it
    • One possible implementation would be to generate the grid and then rotate the coordinate system

2024-01-04

Time: 17:00 UTC

Participants:

  • Santi
  • India
  • Mariana
  • Leo

Quick updates:

  • Santi went to AGU

    • Met Matt Tankersely in person! Matt had a poster, and a lightning talk in the same session as me. Was great to talk about Fatiando and also see his presentation mentioning the same tools.
    • Met Blazej Bucha. He contributed to Boule. Great work on spherical harmonics: https://github.com/blazej-bucha.
    • Lunch with Fernando Perez, Jim (2i2c) and Rowan and Steve (Curvenote)
    • Geolatinas gatherings!
    • Met Yago and Suzana (Yago will start as Master Student with Leo)
    • Met Carlos Alberto Moreno Chaves, from USP.
    • Ines Sanchez Rodriguez, Hydrology. opencollaborationhydrology.com. Got invited to participate in a podcast about open-source. It'll probably be in Spanish.
    • Hélène Le Mével. Awesome inversions of gravity satelite data before, during and after the eruption of Hunga Tonga.
    • Tasha Snow. CryoCloud: https://www.cryocloud.io/. Great talk about why open science!
      • Intelectual generosity
      • Intelectual humility
      • Right to participate in science (democratizing it)
      • Everyone deserves dignity and respect
      • Open Science is a process, not a product!
    • Project Pythia.
    • Center of Open Science: https://www.cos.io/
    • Folks from NumFOCUS were there: Xarray, Dask, etc. Ryan Abernathey, Deepak Cherian.
    • Nigel Rees. Haven't got the chance to meet him, but he emailed me about the Fatiando poster. Matt had a talk with him.
    • Jiajia Sun. Interested in having magnetic equivalent sources in Harmonica.
    • I gave away a lot of Fatiando stickers. Even in the plane: the student that was sitting next to me knew about it :rolling_on_the_floor_laughing:
      • Sarai Montañez Muñoz

  • Faster terrain correction by block averaging.

  • Plan to start adding kernels in Choclo for magnetic gradiometry

  • Discussion on Harmonica Issues for better defaults for equivalent sources:

    • Santi will tackle the removal of the depth_type
    • India will tackle the better defaults for window_size and depth

  • Leo: Started playing with spherical harmonics. SHTools is a bit large and overkill when we just want simple synthesis of a low degree model (like a lot of magnetic models). Plus, there is no package I found that can do the inversion part for scattered data (SHTools only does it for grids). I'll need this for class and for some projects so will start coding something and maybe get a student involved. Should this be part of Harmonica or its own package? The surface harmonics can be used for other things and the associated Legendre function calculation as well (scipy has it but it wraps an ugly fortran code and the normalization isn't very clear). But otherwise, this fits with the equivalent source stuff in Harmonica.

    • It could live in Harmonica!

  • Improving tests for Pooch downloaders:

    • Right now Pooch's tests rely on third-party servers being available when testing
    • If those servers go down, Pooch tests start failing
    • Idea: mock httpservers with pytest-httpserver for testing purposes.
    • Keep the current tests, but maybe make them optional
    • We should open an issue for creating an sftp server using pytest just to test it

  • Design choices for forwards and physical equivalent sources:

    • Merge magnetic forward functions in Harmonica to a single one, that either returns a single array, or a tuple of arrays
      • Those should take a field argument that could be b_e, b_n, b_u or b for the full vector, required argument.
    • Constructing the equivalent sources takes regular parameters plus the inc,dec of the sources
    • Fitting would be nice as ``
    • Design draft for the fit and predict methods for the eq mag sources 
      class EquivalentMagSources:
    def __init__(self, ...):
        ...
        
    def fit(self, tfa=None, b_ee=None, ...):
        ...

    def predict(self, coordinates, field):
        ...
    
eqs = hm.EquivalentMagSources(
    depth=..., 
    inc_sources: float =..., 
    dec_sources: float =...,
)
eqs.fit(
    tfa=(coordinates, data, weights, inclination, declination),
    b_ee=(coordinates, data, weights),
    b_nn=(coordinates, data, weights),
)
b_ee = eqs.predict(coordiantes, data, field="b_ee")
(b_e, b_n, b_u) = eqs.predict(coordiantes, data, field="b")
tfa = hm.total_field_anomaly((b_e, b_n, b_u), inc, dec)

  • We might want to check if hm.magnetic_angles_to_vec work well with inc and dec as floats and amplitude as array.

    • Write tests!

  • The mag forward modelling functions take magnetizations as 2d array with each magnetic moment per row. We should change this to accept a tuple with the three components:

    dipole_magnetic(
    coordinates,
    dipoles,
    magnetic_moments=(magnetization_e, magnetization_n, magnetization_u),
    parallel=True,
    dtype="float64",
    disable_checks=False,
):
    • This makes it consistent with hm.magnetic_angles_to_vec and with how coordinates are handled.

  • Leo: Trying to find a good magnetic microscopy dataset to add to Ensaio. Will clean up the one we used in this paper because it's relatively good with a bunch of well behaved dipoles. Great for running grid-related examples in Harmonica since the data is on a regular grid at constant height from the very start. Would also be good for an Euler Deconvolution example since they're all very close to dipoles.