Add fitting to calibration routines #25

andrea-pasquale · 2022-08-12T13:27:07Z

In this PR I've implemented a first prototype for the post-processing of the data generated by the calibration routines.
It will be sufficient to add the decorator @fit to a calibration routine to do the following:

perform fitting procedures
update the platform runcard (which will be saved in the same directory containing the dataset for that particular routine)

In order to do this I've created a module called fitting which contains two different files:

methods.py which contains routine-specific fitting methods of the form <routine_name>_fit
update.py which contains routine-specific methods for updating the platform runcard of the form <routine_name>_fit

The ActionBuilder will take care of associating each calibration routine with the corresponding methods for fitting and updating the platform runcard. For now, the fitting methods are the same that are used in the branch alvaro/latest of qibolab.

This branch is working with this branch in qibolab, since I am using the latest calibration routines and the new names for the platform runcard which are not been yet pushed to main.

@scarrazza @stavros11 let me know if you like this approach.
It should not be difficult to use the same approach for generating plots.

scarrazza · 2022-08-12T19:11:19Z

@andrea-pasquale thanks, the mechanism looks good to me, please remember to update the sphinx docs accordingly explaining how to implement a complete action.

andrea-pasquale · 2022-08-13T10:21:47Z

Thanks for the feedback @scarrazza.
I believe that we need to port the plotting methods to the decorator mechanism before editing the docs.
So that we can have a final set of instructions that will include coding:

the calibration itself
fitting methods
how to update the platform runcard
which plots should be generated

About the plotting decorators. what did you have in mind?
The most basic thing would be to have a decorator @save_fig that will save the plots generated by the live-plotting.
About the live-plotting we want to automatically generate plots for every routine or do you want to add a generic decorator @plot or @liveplot to select which plots do we want to generate?

scarrazza · 2022-08-15T06:34:04Z

My opinion is the following:

please go ahead and use the decorators names you like. From our perspective these decorators are only instructing the action builder the steps to be execute with the returning data.
all functions associated to decorators of a specific action should be stored in the same file.
I agree with live plotting and report plotting, the first should operate exclusively with qq-live, while the second dumps the final plot that will appear in the report.

maxhant · 2022-08-23T10:39:25Z

@scarrazza
I am not sure what would be the advantage of a decorator rather than a method from the Dataset. In the maxime/ramsey branch, you can find in ramsey.py the function ramsey_frequency_detuned. It is iterating through different scans until it corrects the qubit frequency for s given scan duration to maximize T2. To do that, one needs to fit the curve after completing the first inner loop.
A decorator here would not work as the fitting is used within the function, and we would need to import the fitting function directly. However, this is a special case, and maybe we could be working both ways.

Regarding scans with two variables, how will you be able to fit every row? This comes back to the point I had raised about being able to convert the flat array into a matrix.

andrea-pasquale · 2022-08-27T07:56:15Z

I've implemented the live-fitting for the resonator spectroscopy.
In particular:

The results of the fitting are saved inside the routine directory in a file named fit.yml
When starting qq-live the live-plotting will also show the results of the fitting which is dynamically updated.
Under the plot I've added a few annotations with the relevant parameters found by the fitting, in this case the resonator frequency and the peak voltage, which are also dinamically updated.

This PR have been tested on qpu1q (runcard example1q.yml) and qpu5q (runcard example5q.yml).
Here are a few examples:

Everything is now hardcoded for the resonator spectroscopy, if you are happy with this, I'll start to reorganize the code.
Let me know if this is what you had in mind for the livefitting @DavidSarlle @maxhant @aorgazf @scarrazza @stavros11 .

stavros11

Thanks for implementing this. I like the idea of adding the live fitting line to the live plotting, but let's also hear David's and Maxime's opinion as the main users of this.

If fitting is fast for all actions, I do not see any issue with putting it on all live plots and reports. On the other hand, if fitting is causing any bottleneck, we should consider having a way to switch it on or off (perhaps through an option in qq or qq-live) in case someone wants to run a scan without fitting.

stavros11 · 2022-08-28T18:09:16Z

src/qcvv/fitting/utils.py

+def get_values(df, quantity, unit):
+    return df[quantity].pint.to(unit).pint.magnitude


In #26 which is already merged to add_routines I added this as a method to the Dataset object, because it is also useful for plotting. So you can probably remove this from here and just do data.get_values when you use it in the fitting methods.

I also like the idea of the live fitting. As @stavros11 pointed out, if there is no bottleneck due to the live fitting, I really like the idea. If it is not the case, I agree in the idea of having a way to switch it on and off.
Thanks @andrea-pasquale for implementing this functionality.

In #26 which is already merged to add_routines I added this as a method to the Dataset object, because it is also useful for plotting. So you can probably remove this from here and just do data.get_values when you use it in the fitting methods.

Thanks, I forgot to remove it here.

stavros11 · 2022-08-28T18:21:58Z

src/qcvv/decorators.py

    f.prepare = prepare_path
    f.final_action = save
    return f
+
+
+def fit(f):


At this stage, I am not really sure if the store and fit decorators are the best way to go, especially if we end up using them in all calibration methods. To me, it seems easier to have store and fit enabled for all calibrations, for example through the action builder, without having to explicitly use @store and @fit in every method. I may be missing something so let's see what others think.

In #45 I implemented a plot decorator but this is different as it can be used to map multiple calibration methods to the same plotting function. This is relevant for the calibrations added by Maxime and David. Perhaps when you implement fit for more calibrations, something similar may be useful (eg. @fit() with arguments). But at least for store, it seems that it is the same for all calibration methods and it is also essential for all other features (fitting, report, live plotting), so we could consider hard-coding it to the ActionBuilder and removing the decorator.

Also agree with @stavros11. At the end, we are going to store and fit all the data obtained from the calibration methods. To me the @store and @fit decorators should be enabled in all calibrations by default.

Regarding #45, this functionality is also important, because until now I had to implement a method for plotting for each calibration routine even if they were more or less the same (f.e when only units changed, I had to code a new function).
Thanks @stavros11

Thanks for the feedback. I also agree that at this point especially store should not be a decorator, given that the (live)plotting is reading the data directly from file. Regarding the fitting I think that we can follow a similar approach for the plotting. We can always perform the fitting and save the results in the report. Then we can have a decorator in order to perform the live-fitting which should be optional.

stavros11 · 2022-08-28T18:29:55Z

src/qcvv/web/app.py

+        if len(data) > 2:
+            params, fit = resonator_spectroscopy_fit(folder, format, nqubits)
+            return getattr(plots.resonator_spectroscopy, method)(data, params, fit)


This could just be:

Suggested change

if len(data) > 2:

params, fit = resonator_spectroscopy_fit(folder, format, nqubits)

return getattr(plots.resonator_spectroscopy, method)(data, params, fit)

if len(data) > 2:

params, fit = resonator_spectroscopy_fit(folder, format, nqubits)

else:

params, fit = None, None

return getattr(plots.resonator_spectroscopy, method)(data, params, fit)

and have a single return.

Also, keep in mind that this method is out-dated in this branch, as we have been updating it with Maxime as he was adding more calibrations in his branches. I would say the latest stable version that works on the actual experiments is in maxime/routines branch and I simplified it a bit more in #45 where I removed plots.yml. The important thing is that when you generalize this to arbitrary actions, you will need a getattr-like approach to call the proper fitting method for your action.

stavros11 · 2022-08-28T18:45:20Z

src/qcvv/web/app.py

+        with open(f"{folder}/platform.yml", "r") as f:
+            nqubits = yaml.safe_load(f)["nqubits"]


This method is called automatically every second by dash to update the figures, so we should optimize it as much as possible. For example here we could pass nqubits (assuming its a constant) through the url, similarly to routine, format, etc., which is rendered once during the first page load, to avoid reading from the platform.yml from disk every second.

I completely agree, this was just a temporary solution to have things working.

stavros11 · 2022-08-28T18:49:49Z

src/qcvv/web/app.py

+        with open(f"{folder}/platform.yml", "r") as f:
+            nqubits = yaml.safe_load(f)["nqubits"]
+        if len(data) > 2:
+            params, fit = resonator_spectroscopy_fit(folder, format, nqubits)


Similarly to the comment above, here you are fitting the data in every live plot update. Would it be easier to do the fit within the action, dump the results on disk and just load them here for plotting (similar to what we do with data)?

I think that as it is, given that the live plot is typically updated more frequently than the actual data, this will perform the same fit multiple times before new data are available.

I think that this could be a good idea, given that some of the calibrations will also require fitting inside the routine itself and not as a post-processing step.

I think that as it is, given that the live plot is typically updated more frequently than the actual data, this will perform the same fit multiple times before new data are available.

This is true. If we decide to include the fitting inside the calibration we should then be able to call the fitting using a parameter similar to points to decide when we should perform the fitting and save it to file.

stavros11 · 2022-08-28T18:51:06Z

src/qcvv/plots/resonator_spectroscopy.py


-class resonator_spectroscopy_attenuation:
+
+class resonator_spectroscopy:


This is also out-dated. In maxime/routines we simplified the plots to functions instead of classes, as the same plot function can be re-used in multiple routines.

andrea-pasquale · 2022-08-30T07:28:38Z

Thanks for the review and the comments @DavidSarlle @stavros11.
As already discussed in the comments I think that the easiest solution for the fitting is to perform the fitting inside the calibration routine itself, let me know if you agree @scarrazza.
I will try to rebase this to #45 and to implement all the modifications suggested. If it becomes too messy I will open another PR and close this one.

scarrazza · 2022-08-30T07:32:53Z

I think that the easiest solution for the fitting is to perform the fitting inside the calibration routine itself, let me know if you agree @scarrazza.

okay, sounds reasonable.

andrea-pasquale · 2022-08-31T16:48:55Z

Now this PR works with the @plot decorator implemented in #45.

I've implemented the following changes:

Enable @store in all calibration routines by removing the decorator.
Implement the (live)-fitting without the decorator @fit
Generate updated platform runcard at the end of the calibration routines.

Regarding 2 the fitting is now performed directly inside the routine. As @stavros11 pointed out, in order to avoid unnecessary calls to the fitting, the fitting is performed the same time that we store to file using the parameter points. I can also implement a different parameter instead of points but to me it seems a bit redundant.
To store data from the fitting I've implemented a new class Data which removes pint from the Dataset, which sometimes can be tricky to use if the user wants to save non-physical quantities, like the results of a fit.
I've not implemented a custom decorator for the live-fit, it will be sufficient to use the @plot decorator. I can also implement another decorator for the live-fitting, let me know what you think.
For now I've added the fitted only the resonator spectroscopy, you can test this PR using the runcards example1q.yml and example5q.yml. Compared to before, it will be much easier to add the fitting to other routines.
Let me know what you think @scarrazza @stavros11.

andrea-pasquale · 2022-09-01T12:10:12Z

I've also added the fitting on the qubit spectroscopy.
Probably there is something wrong on qpu5q, but we can fix this later. The fitting is working as expected.
I'm not planning to add something else to this PR.

scarrazza · 2022-09-01T17:05:36Z

@stavros11 can we merge this now?

andrea-pasquale added 2 commits August 12, 2022 17:03

Implement fitting module

3a54a85

Add fitting module

c0c3802

andrea-pasquale requested review from scarrazza and stavros11 August 12, 2022 13:27

Merge branch 'add_routines' into fitting

99c37d8

stavros11 mentioned this pull request Aug 25, 2022

Pass multiple qubits in the action runcard #33

Merged

andrea-pasquale added 5 commits August 25, 2022 13:57

Fix fitting for resonator spectroscopy

2c2f38c

Implement live fitting for qpu1q

b4d7ea5

Fix live-fitting for qpu5q

6efc499

Enable fitting

ed407cb

Add qpu5q runcard

d45b0b0

stavros11 reviewed Aug 28, 2022

View reviewed changes

andrea-pasquale added 2 commits August 30, 2022 10:59

Implement Stavros suggestions

347fce2

RFix from previous commit

9b569dd

andrea-pasquale added 3 commits August 30, 2022 18:07

Merge branch 'plotdecorator' into fitting

7d7c5da

Remove store decorator

53c1693

Remove qq-live warning

9ae838b

stavros11 mentioned this pull request Aug 30, 2022

Add plot decorator #45

Merged

andrea-pasquale added 4 commits August 31, 2022 16:59

Implement fitting and store results with Dataset class

db19774

Implement live-fittting and Data class

c303a99

Add annotations

6986034

Add updated platform runcard at the end of a routine

b6b9fb0

andrea-pasquale changed the title ~~Add fit decorator~~ Add fitting to calibration routines Aug 31, 2022

andrea-pasquale changed the base branch from add_routines to plotdecorator August 31, 2022 16:21

Updated runcard

c156a9e

Generalize fitting for qubit spectroscopy

8f15686

scarrazza merged commit 9fe7b23 into plotdecorator Sep 1, 2022

scarrazza deleted the fitting branch October 10, 2022 18:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add fitting to calibration routines #25

Add fitting to calibration routines #25

andrea-pasquale commented Aug 12, 2022

scarrazza commented Aug 12, 2022

andrea-pasquale commented Aug 13, 2022

scarrazza commented Aug 15, 2022

maxhant commented Aug 23, 2022

andrea-pasquale commented Aug 27, 2022

stavros11 left a comment

stavros11 Aug 28, 2022

DavidSarlle Aug 29, 2022

andrea-pasquale Aug 30, 2022

stavros11 Aug 28, 2022

DavidSarlle Aug 29, 2022

andrea-pasquale Aug 30, 2022

stavros11 Aug 28, 2022

andrea-pasquale Aug 30, 2022

stavros11 Aug 28, 2022

andrea-pasquale Aug 30, 2022

stavros11 Aug 28, 2022

andrea-pasquale Aug 30, 2022

stavros11 Aug 28, 2022

andrea-pasquale commented Aug 30, 2022

scarrazza commented Aug 30, 2022

andrea-pasquale commented Aug 31, 2022

andrea-pasquale commented Sep 1, 2022

scarrazza commented Sep 1, 2022

		def get_values(df, quantity, unit):
		return df[quantity].pint.to(unit).pint.magnitude

		with open(f"{folder}/platform.yml", "r") as f:
		nqubits = yaml.safe_load(f)["nqubits"]


		class resonator_spectroscopy_attenuation:

		class resonator_spectroscopy:

Add fitting to calibration routines #25

Add fitting to calibration routines #25

Conversation

andrea-pasquale commented Aug 12, 2022

scarrazza commented Aug 12, 2022

andrea-pasquale commented Aug 13, 2022

scarrazza commented Aug 15, 2022

maxhant commented Aug 23, 2022

andrea-pasquale commented Aug 27, 2022

stavros11 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andrea-pasquale commented Aug 30, 2022

scarrazza commented Aug 30, 2022

andrea-pasquale commented Aug 31, 2022

andrea-pasquale commented Sep 1, 2022

scarrazza commented Sep 1, 2022