-
Notifications
You must be signed in to change notification settings - Fork 578
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WIP: gradient support #25
Comments
It seems the gradient support is recently addressed :-) |
Not quite :-) Still waiting on it. |
Added the basic binding, but I have not written the tests, I updated the bug description to track that. Additionally, the tensorflow commit [0] not all capabilities from the C++ API have been surfaced yet. |
Is it now possible to retrieve the gradients and do some form of gradient descent? |
I would like to ask the same question - using the new pre-release package that has just been uploaded to NuGet (1.3.0-pre1), is it already possible to retrieve gradients for tensors and do any form of gradient descent at this time? I am not totally currently up-to-date with the status of the C API support for this feature yet, so I guessed it would be easier to ask :-) |
@migueldeicaza Hi, I started to verify AddGradients() API with code like this:
and got couple of problems: but the API returns only one result 6. 1 is quite simple to fix |
Hi @Dorokhov, If I understood correctly TensorFlow's documentation for AddGradients, the return vector should have the same length as the inputs vector, therefore the answer would have indeed just one value (cf. the doc: "The partial derivatives are returned in But maybe I am wrong, I am just starting with the gradients API. Let me see if I can help find where the problem is. Regards, |
Hi @cesarsouza, I think you are right, and the API should produce one value - partial derivatives of 'y' sums which are 17502 (6+17496) in my case, but it returns 6. I will try to test the same case in native API. Thanks. |
I'm struggling with getting simple models built in Python "productionized" into C#. Take a model built, trained, and saved in python. The most challenging part for me is constructing the InputVariables, InputValues, and Output. The construction patterns, and instance usages for those objects are befuddling me to say the least. I assume I am not seeing a pattern in the examples, or that I am fighting the naming conventions, or... I don't know what I am missing... And I can't be the only person struggling with this. The python code is attached. It's a simple linear regression model I ripped off from a stanford class and made more portable. Here are the samples i am trying to create. Happy to donate them to the cause when they are complete. |
Hi @sqlBender, I have to say I also share your pain, but I am not sure if the (actually very relevant) issue you have raised is connected to the original topic of this current issue, that is, the ability to obtain automatic gradient calculations through the AddGradients method. If you want to take a look, the Keras Sharp project is aiming at providing an API that is very similar to its Python equivalent, but unfortunately that project is also still a bit blocked until this very issue here gets eventually addressed. Regards, |
@cesarsouza it's just where I found an issue similar. Gradient etc. I'll spin up a new issue since my issue is not really related to this thread. |
Started discussing the gradient API issue in the tensorflow repository, hoping we will find how the API actually works soon. |
These changes and the fix in native API resolves issue #25 Now, add gradients works correctly The test will fail until a new version of TensorFlowSharp with updated native libraries is released.
Seems like TF doesn't have all gradient operations defined yet. I am referencing here an issue I've just created in TF's issue tracker regarding a missing gradient for tf.select. |
Hi @migueldeicaza, @cesarsouza, The Piece of code // tf Graph Input
var X = tf.placeholder(tf.float32);
var Y = tf.placeholder(tf.float32);
// Set model weights
// We can set a fixed init value in order to debug
// var rnd1 = rng.randn<float>();
// var rnd2 = rng.randn<float>();
var W = tf.Variable(-0.06f, name: "weight");
var b = tf.Variable(-0.73f, name: "bias");
// Construct a linear model
var pred = tf.add(tf.multiply(X, W), b);
// Mean squared error
var cost = tf.reduce_sum(tf.pow(pred - Y, 2.0f)) / (2.0f * n_samples);
// gradient descent
// Note, minimize() knows to modify W and b because Variable objects are trainable=True by default
var optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost);
// Initialize the variables (i.e. assign their default value)
var init = tf.global_variables_initializer(); |
This requires the C API to get support for it.
There is a bug here: tensorflow/tensorflow#6268
The text was updated successfully, but these errors were encountered: