New data generators #34

andrevitorelli · 2021-10-21T11:09:08Z

I am trying to generate a more realistic set of galaxy stamps to test autometacal. These methods will be available inside the datasets subpackage and eventually integrated into tensorflow-datasets api within autometacal.

I have created a notebook at notebooks/Datasets.ipynb to explore different ways to do so.

Currently, I have implementations of:

A simple exponential profile convolved with a moffat PSF
Creating parametric galaxies from the COSMOS dataset
Cutting out galaxy stamps directly from the COSMOS dataset

Both 2. and 3. have problems for sure, which is why I'm seeking some guidance from @aguinot also. I'm trying to improve 2. getting code from galaxy2galaxy now.

PS: I have also tried to do bulge+disk models (with some reasonable assumptions) by modifying 1., and tested them as I was doing before - the results were comparable to 1.

updating mcal auto/finite test with latest version of mcal img gen

Updating working branch

Adding test notebook

aguinot · 2021-11-04T10:52:00Z

autometacal/python/datasets/galaxies.py

+  gal_image = obs.drawImage(nx=defaults['stamp_size'], 
+                            ny=defaults['stamp_size'], 
+                            scale=defaults['scale'])
+  noise = galsim.GaussianNoise()


It could be nice to give an instance of galsim.BaseDeviate() here, with you would initialize with a seed that you take as input. That would help the reproducibility.

Yes, I do plan to do this next.

aguinot · 2021-11-04T10:52:59Z

autometacal/python/datasets/galaxies.py

+  }
+
+  disk_frac   = 1 - bulge_frac
+  smooth_disk_frac = 1 - knot_disk_frac 


Some ref/explanation would be nice here as well.

aguinot · 2021-11-04T10:53:49Z

autometacal/python/datasets/galaxies.py

+  disk_hlr = .7,
+  bulge_frac = 0.4,
+  knot_disk_frac = 0.7,
+  n_knots = 100,


That is a very nice addition. Do you have a ref or a justification for those numbers ?

aguinot · 2021-11-04T10:55:15Z

autometacal/python/datasets/galaxies.py

+  knotted_disk = galsim.RandomKnots(n_knots, 
+                                    half_light_radius=disk_hlr, 
+                                    flux=knot_disk_frac,) 
+                                    #rng=rng)


I would definitely not remove the rng initialization. It is something that you really want to do.

aguinot · 2021-11-04T10:55:34Z

autometacal/python/datasets/galaxies.py

+                            scale=defaults['scale'])
+  gal_image = obj.drawImage(nx=defaults['stamp_size'], 
+                            ny=defaults['stamp_size'], 
+                            scale=defaults['scale'])
  noise = galsim.GaussianNoise()


Same here, seed.

aguinot · 2021-11-05T11:04:58Z

autometacal/python/moments.py

+
+  q1 = Q11 - Q22
+  q2 = 2*Q12
+  T = Q11  + Q22 + 2*tf.math.sqrt(tf.math.abs(Q11*Q22-Q12*Q12))


Not a big fan of this definition.. Having the sqrt is risky. I am pretty sure that ngmix doesn't use this definition either.

I had changed to the ngmix definition (T = Q11 + Q22), but since GalFlow uses this one, I switched it back.

Hummmmm..... it's not that GalFlow uses this one, it's just that in GalFlow we apply a shear g, which is different than e. I've proposed reverting back to T = Q11 + Q22 in #37

aguinot · 2021-11-05T12:31:51Z

autometacal/python/moments.py

+  q1 = Q11 - Q22
+  q2 = 2*Q12
+  T = Q11  + Q22 + 2*tf.math.sqrt(tf.math.abs(Q11*Q22-Q12*Q12))
+  result = tf.stack([q1/T, q2/T], axis=-1)[0]


Having the size (T) in the output could be very nice!
Maybe we should use another definition of the size but T is fine for now.

Yes, that's true, but we can make another PR for that, this is out of the scope of this one.

aguinot · 2021-11-05T12:42:15Z

tests/test_tf_ngmix.py

+  gals, _ = autometacal.datasets.galaxies.make_data(Ngals=Ngals, snr=100,
+                                                 gal_g1=np.random.uniform(-.7,.7,Ngals),
+                                                 gal_g2=np.random.uniform(-.7,.7,Ngals),
+                                                 scale=scale)


In my opinion, you should really have a seed set here (for np.random but also inside make_data for the noise in GalSim. By the way, since you are using gaussian noise, you could do it with NumPy so you only have to initialize NumPy). It is very important that each time we run this test we get the exact same result. Also, it would be nice to know the SNR used here for those tests.

aguinot · 2021-11-05T12:43:59Z

tests/test_tf_ngmix.py

+  weights = autometacal.tf_ngmix.create_gmix([0.,0.,0.,0.,T,1.],'gauss')
+  result_tf_ngmix = autometacal.tf_ngmix.get_moments(weights,pixels)
+
+  assert_allclose(results_ngmix,result_tf_ngmix,rtol=1e-6,atol=1e-6)


Based on your definition of ellipticity (different from ngmix), I am very surprise that this test pass. Maybe if you use a gaussian profile both definition (chi/epsilon) are equivalent... This would need to be investigated.

I think this is missing an update from the tf_ngmix branch, I changed it by first applying e1e2_to_g1g2() from ngmix before comparison.

aguinot · 2021-11-05T12:47:09Z

tests/test_tf_ngmix.py

+def test_tf_ngmix():
+  """
+  This test generates a simple galaxy and measure moments with ngmix, vs.
+  tf_ngmix.


Sorry but those tests are important in my opinion, and they would required a bit more explanation. Like what is a "simple galaxy"? (SNR as well)

aguinot · 2021-11-05T14:35:47Z

In addition to the comment I let on the code, here are some comments on the notebooks:

Dataset.ipynb:
- Why you do not use the function in galaxy.py?
- Also we should discuss about the tests we want to make because having a fix SNR and a flux that vary is not very realistic. Basically, if the SNR is fixed varying the flux will not change anything in the results I think.
- In Real Images - I I feel like some galaxies looks "weird", maybe it is juste me.. The noise looks strange as well.
- In Real Images II I feel like there is some patterns in the noise?
metacal_comparison.ipynb:
- In Test GalFlow Deconv/Reconv are the residual actually small? what is the maximum pixel value in your original image? Because if the flux is 1, like it is in your default, this is actually not that "small" a few percent. But maybe it is expected?
- In GalFlow vs GalSim same comment ^^^^^
- I don't understand those plots: reconv w/ same psf noshear
- In Simple Ellipticity Measurements I don't like what I see here. The value you measure are "far" from the expected one. Maybe this is because of the step size, could you re-try with step=0.01?
- When you print the calibrated e1/e2 it could be nice to have an idea of the variance to see if the measured residuals are significant or not. If not, then we should increase the number of galaxies (or increase the SNR). To give you an idea in the unit test used to check metacal in the ngmix repo they use 1000 galaxies with SNR=80000.
- I am still confuse by the discrepancy between R11/R22. I would be curious to see what ngmix gives you on the same sample.
- In Averaging the response comparison over many galaxies you should fix the plot, we cannot see what is going on. Also, I am very surprise by the value you have (-350), this should not happen given your dataset.
- The end of your code failed

Some general comments:

It is hard to check all the changes, there is a lot of files
Some of the files are more tan a month old
Some clean up would be nice :)
Globally I think it is pretty nice and we are getting closer, it is very nice to see!

aguinot · 2021-11-05T14:37:10Z

autometacal/python/datasets/galaxies.py

+    'scale' : 0.2,
+    'stamp_size' : 51,
+    'psf_fwhm' : 0.9, 
+  }


This could be set as a "global" variable since it is used in all of your functions

EiffL · 2021-11-06T20:38:37Z

I've updated this branch to main.

andrevitorelli · 2021-11-18T10:01:37Z

This PR should be dropped as #40 covers it.

andrevitorelli and others added 30 commits June 22, 2021 09:52

simple mcal test nb

34e2b8e

smaller ellips, metacal tests

8c6707e

Merge pull request #21 from CosmoStat/main

46ff248

updating mcal auto/finite test with latest version of mcal img gen

comparing autodifferentiation and finite differences

7002e24

Delete metacal_compare_auto_finite.ipynb

5e5e0a8

comparison auto finite differentiations

e5a4375

notebook work

d52b410

minor ch

8ae483b

averaging over many galaxies

d5cf9ff

further investigations at notebook

dfda012

prototyping

708b20b

Add files via upload

056faf2

Merge pull request #24 from CosmoStat/main

430167e

Updating working branch

nightly work

d4e7c16

nightly work

9d71697

daily

fa28761

daily work

f6336c1

daily work

3a52646

Adding test notebook

226545c

Merge pull request #27 from CosmoStat/u/EiffL/test_grads

39879be

Adding test notebook

notebook work

160ad04

adding functions for ellipticities, finite diferences to the package

9895dd4

name change, tf_ngmix initial work

6324cf5

name change, tf_ngmix initial work

30a95ed

restructuring

6e7c459

tf ngmix gaussmom implementation

b82d393

adding the tf_ngmix test file

9bf09a9

tf_ngmix

6ece5e4

tf_ngmix

bc89e35

removing previous tf_ngmix work

c845c83

andre zamoranovitorelli and others added 12 commits October 12, 2021 16:44

datasets

ff52953

new dataset work

cc1adaf

new dataset work

85ceb6e

new dataset work

2f552dd

Merge branch 'u/andrevitorelli/restructure' into dataset

7747aa0

working on making simulated galaxies

a7fa238

new datasets work/tests

eccb4b8

working on datasets

6532fbe

getting new moments

c92441e

dataset work

de5a5ee

dataset work

a962dd7

developing datasets

8faa22c

andrevitorelli requested review from EiffL and aguinot October 21, 2021 11:09

andrevitorelli added 7 commits October 21, 2021 13:50

removing old notebooks

9778417

datasets

f33078d

fixing moments bug (must be g, not e, by galsim defs)

c427037

dataset work

296b268

work on datasets

cd8d2d9

dataset work

f394b48

autodiff-finitediff work

dbd6a1f

aguinot reviewed Nov 5, 2021

View reviewed changes

EiffL mentioned this pull request Nov 6, 2021

Build a realistic galaxy sample, with some typical SNR, shape, size #39

Closed

EiffL self-assigned this Nov 6, 2021

Merge remote-tracking branch 'origin/main' into dataset

18bd037

andrevitorelli closed this Nov 18, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New data generators #34

New data generators #34

andrevitorelli commented Oct 21, 2021

aguinot Nov 4, 2021

andrevitorelli Nov 5, 2021

aguinot Nov 4, 2021

aguinot Nov 4, 2021

aguinot Nov 4, 2021

aguinot Nov 4, 2021

aguinot Nov 5, 2021

andrevitorelli Nov 5, 2021

EiffL Nov 6, 2021

aguinot Nov 5, 2021

EiffL Nov 6, 2021

aguinot Nov 5, 2021

aguinot Nov 5, 2021

andrevitorelli Nov 5, 2021

aguinot Nov 5, 2021

aguinot commented Nov 5, 2021

aguinot Nov 5, 2021

EiffL commented Nov 6, 2021

andrevitorelli commented Nov 18, 2021

New data generators #34

New data generators #34

Conversation

andrevitorelli commented Oct 21, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aguinot commented Nov 5, 2021

Choose a reason for hiding this comment

EiffL commented Nov 6, 2021

andrevitorelli commented Nov 18, 2021