added additional loss against data for NNODE #666
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This looks good, can you write some tests to ensure its working? We need to test with each strategy. We will need to do documentation too. Where in the paper is this additional loss mentioned? |
|
in the paper the loss term has additional loss terms :Lgls and Lconstr(under introduction eq 3 to 8), Lgls is defined against cell density predictions via a Surrogate Model MLP which takes in data as measured u at (x,t), Lconstr performs in parameter Estimation for the Growth, Time Delay and Diffusion terms(parameter functions are functions of the solution itself -> leads to issue #572). |
|
Ah I see it, yep this looks like it implements what's missing, good job 👍 |
Open
|
The test failure looks real. |
src/ode_solve.jl
Outdated
Comment on lines
298
to
311
| function generate_loss(strategy::WeightedIntervalTraining, phi, f, autodiff::Bool, tspan, p, | ||
| batch) | ||
| minT = tspan[1] | ||
| maxT = tspan[2] | ||
|
|
||
| weights = strategy.weights ./ sum(strategy.weights) | ||
|
|
||
| N = length(weights) | ||
| samples = strategy.samples | ||
|
|
||
| difference = (maxT - minT) / N | ||
|
|
||
| data = Float64[] | ||
| for (index, item) in enumerate(weights) | ||
| temp_data = rand(1, trunc(Int, samples * item)) .* difference .+ minT .+ |
There was a problem hiding this comment.
Rebase issue? Rebase to the new master.
|
Rebase this onto the new master so that it's not dependent on any other PR (it's now merged). |
(user defined function), also now has common OptimizationFunction object definition(instead for each TrainingStrategy).
Also updated docs(might need further editing) Future Scope: We can add weighted loss in NNODE,and even losses for Parameter Estimation for Inverse Problems.
(user defined function), also now has common OptimizationFunction object definition(instead for each TrainingStrategy).
Also updated docs(might need further editing) Future Scope: We can add weighted loss in NNODE,and even losses for Parameter Estimation for Inverse Problems.
(user defined function), also now has common OptimizationFunction object definition(instead for each TrainingStrategy).
Also updated docs(might need further editing) Future Scope: We can add weighted loss in NNODE,and even losses for Parameter Estimation for Inverse Problems.
(user defined function), also now has common OptimizationFunction object definition(instead for each TrainingStrategy).
Also updated docs(might need further editing) Future Scope: We can add weighted loss in NNODE,and even losses for Parameter Estimation for Inverse Problems.
src/ode_solve.jl
Outdated
Comment on lines
421
to
426
| # additional loss | ||
| additional_loss = alg.additional_loss | ||
|
|
||
| # Creates OptimizationFunction Object from total_loss | ||
| function total_loss(θ, _) | ||
| L2_loss = generate_loss(strategy, phi, f, autodiff, tspan, p, batch)(θ, phi) |
There was a problem hiding this comment.
Suggested change
| # additional loss | |
| additional_loss = alg.additional_loss | |
| # Creates OptimizationFunction Object from total_loss | |
| function total_loss(θ, _) | |
| L2_loss = generate_loss(strategy, phi, f, autodiff, tspan, p, batch)(θ, phi) | |
| inner_f = generate_loss(strategy, phi, f, autodiff, tspan, p, batch) | |
| additional_loss = alg.additional_loss | |
| # Creates OptimizationFunction Object from total_loss | |
| function total_loss(θ, _) | |
| L2_loss = inner_f (θ, phi) |
src/ode_solve.jl
Outdated
Comment on lines
434
to
445
| opt_algo = if strategy isa QuadratureTraining | ||
| Optimization.AutoForwardDiff() | ||
| elseif strategy isa StochasticTraining | ||
| Optimization.AutoZygote() | ||
| elseif strategy isa WeightedIntervalTraining | ||
| Optimization.AutoZygote() | ||
| else | ||
| # by default GridTraining choice of Optimization | ||
| # if adding new training algorithms we can extend this, | ||
| # if-elseif-else block for choices of optimization algos | ||
| Optimization.AutoZygote() | ||
| end |
There was a problem hiding this comment.
Suggested change
| opt_algo = if strategy isa QuadratureTraining | |
| Optimization.AutoForwardDiff() | |
| elseif strategy isa StochasticTraining | |
| Optimization.AutoZygote() | |
| elseif strategy isa WeightedIntervalTraining | |
| Optimization.AutoZygote() | |
| else | |
| # by default GridTraining choice of Optimization | |
| # if adding new training algorithms we can extend this, | |
| # if-elseif-else block for choices of optimization algos | |
| Optimization.AutoZygote() | |
| end | |
| opt_algo = if strategy isa QuadratureTraining | |
| Optimization.AutoForwardDiff() | |
| else | |
| Optimization.AutoZygote() | |
| end |
test/NNODE_tests.jl
Outdated
| @test sol.errors[:l2] < 0.5 | ||
| @test sol.errors[:l2] |
src/ode_solve.jl
Outdated
| @@ -3,7 +3,7 @@ abstract type NeuralPDEAlgorithm <: DiffEqBase.AbstractODEAlgorithm end | |||
| """ | |||
| ```julia | |||
| NNODE(chain, opt=OptimizationPolyalgorithms.PolyOpt(), init_params = nothing; | |||
| autodiff=false, batch=0, kwargs...) | |||
| autodiff=false, batch=0,additional_loss=nothing,kwargs...) | |||
There was a problem hiding this comment.
Suggested change
| autodiff=false, batch=0,additional_loss=nothing,kwargs...) | |
| autodiff=false, batch=0,additional_loss=nothing, | |
| kwargs...) |
src/ode_solve.jl
Outdated
Comment on lines
28
to
33
| example: | ||
| ts=[t for t in 1:100] | ||
| (u_, t_) = (analytical_func(ts), ts) | ||
| function additional_loss(phi, θ) | ||
| return sum(sum(abs2, [phi(t, θ) for t in t_] .- u_)) / length(u_) | ||
| end |
There was a problem hiding this comment.
This isn't in the right spot. Make an example section.
There was a problem hiding this comment.
under this argument's description? or along with the lower example?
src/ode_solve.jl
Outdated
Comment on lines
277
to
278
|
|
||
| return loss |
There was a problem hiding this comment.
Suggested change
| return loss | |
| return loss |
src/ode_solve.jl
Outdated
Comment on lines
293
to
294
|
|
||
| return loss |
There was a problem hiding this comment.
Suggested change
| return loss | |
| return loss |
ChrisRackauckas
approved these changes
Apr 4, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
added option for additional loss against data for NNODE(used PhysicsInformedNN as a reference),work in progress as im getting some errors would greatly appreciate any help. Thanks
. (trying to solve issue #640, but NNODE dosent allow additional losses against surrogate Model predictions nor does it allow parameter estimation so this pr could help solve such ODE problems)