Local development story? #223
Comments
|
There's no story Jimmy. Either MSDN or another Azure subscription can be used to run on the real thing. |
|
So what's the story there then? Each developer creates their own namespace? Picks an option that keeps them under the MSDN limit? Or runs a shared namespace amongst the team? That would run into all the normal problems of shared dev resources. |
|
Either way works.
Shared namespace is fine if you can "sandbox" development. I always advise
to leverage MSDN subscription, unless you plan to use premium. Then you
don't want create that namespace and keep around if not used.
Depending on your scenario I'd have other options, but you know, "it
depends" 🙂
|
|
Or, abstract your service bus layer, and run whatever you want locally?
∞ Andy Badera
…On Wed, Aug 15, 2018 at 9:39 AM, Sean Feldman ***@***.***> wrote:
Either way works.
Shared namespace is fine if you can "sandbox" development. I always advise
to leverage MSDN subscription, unless you plan to use premium. Then you
don't want create that namespace and keep around if not used.
Depending on your scenario I'd have other options, but you know, "it
depends" 🙂
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#223 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAAzzIzJgTGFjCOzulz8FJui2UdFLfuvks5uRDKzgaJpZM4V-M3W>
.
|
|
Layer of abstraction for code, yes. Running code w/o exercising client/broker, no. |
|
How is an abstraction layer any different? Develop locally against whatever
you want, run your automated integration tests against ASB. At a developer
level, it shouldn't matter.
∞ Andy Badera
…On Wed, Aug 15, 2018 at 10:09 AM, Sean Feldman ***@***.***> wrote:
Layer of abstraction for code, yes. Running code w/o exercising
client/broker, no.
When working with ASB, you really should experience the "real thing".
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#223 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAAzzME33C1xxdJ_xe5-hPCdTyDLsCN8ks5uRDmRgaJpZM4V-M3W>
.
|
|
@andrewbadera and when integration test (if you have one) against ASB is failing, what are you going to do locally? Wouldn't you switch back to ASB from whatever you were running against? And now you have to deal with the "perks" of two different technologies. |
|
@andrewbadera don't get me wrong. I am not against layer of abstraction. I'm not a fan of replacing ASB with some other technology to emulate it. |
|
Yeah, there are real differences between different messaging transports that directly affect development. I wouldn't want to swap for example ASB for "messages are files in a folder" |
|
This question is probably related to this feedback where @SeanFeldman also commented. We do have the ability to run everyting dotnet core, redis, functions, sql server, storage and cosmosdb locally, but not servicebus, hence the number of votes on the feedback page. And the Service Bus On-Prem option suggested in the feedback doesn't really seem to be a in-sync option |
|
Yeah, just gonna put my voice here as well. No option isn't really a great selling point. This has downstream effects too with EventHubs which are incredibly useful, but at least I can simulate them by using Kafka protocol enabled hubs, but that pushes me away from functions. My option here is to have every dev in my org use their MSDN account to setup dev resources? Very tempting reason to not use Service Bus and go to Rabbit. |
|
Thoughts @djrosanova ? |
|
It seems like utter nonsense to not be able to develop and run integration tests locally. At present I have to strip out and service bus bindings to test. |
|
@arjunsol Can you use RabbitMQ for localhost dev and switch over to ASB for staging/prod ? Absolute PITA but ... a workaround. |
|
@PureKrome do you mean for mocking purpose? |
|
@ggirard07 code has some abstraction for
Yes. Which is why i'm screaming internally. But when you have to ship something, you gotta work with the tools around you. So to me, switching between the two is far from ideal BUT it works. I got this from MS's NET-Microservices-Architecture-for-Containerized-NET-Applications-(Microsoft-eBook). Also, I tried asking ClemensV about this on twitter a few days ago. Either I failed to communicate my question or we have a different opinion, on the matter. ... especially when there's emulators and images for Azure Storage / Sql or NoSql db's / Redis / Elastic Search / etc.. SIDE NOTE: also like how Udi Dahan said they use Sql Server as their EB for local/test with NServiceBus.SqlServer as their transport/too. cc @jbogard - Can you fill us in, with how you/your team handled this scenario, plz? |
|
It would probably be a lot closer to use a localhost AMQP1.0 broker for the mock @PureKrome ? Unless you were already thinking of the AMQP1.0 support in RabbitMQ server. There's another twitter thread discussing this topic here: https://twitter.com/jeremydmiller/status/1080496980168712192 |
Um ... er .. all i've researched is using RabbitMQ and have no idea (yet) what this means with resepct to protocols/etc. All this time spent is being wasted (unfortunatly) getting the tooling and abstractions to work ... just to get a localhost dev story up-n-running. Man - this is 2019 and this should be a solved problem already :( These tools have been around for a while now. So much frustration :( All I know is this:
so - frustrating :( |
|
I had thought of that, we're using FunctionMonkey to cut down on boiler plate. I'm unsure if I can make it play nicely. |
|
It's possible to trigger a service bus trigger via http using the /admin route, (https://docs.microsoft.com/en-us/azure/azure-functions/functions-manually-run-non-http) though I haven't quite got the hang of serializing to a BrokeredMessage, this would be step 1 for me, as I could then just trigger my function via postman when running locally. The second issue is that if you have a service bus triggered function and you don't provide a connection string the trigger will blow up. If there was a way round this that would be great. Maybe being able to switch out the type of trigger via configuration or suppressing the exception, but not sure if that's possible. Anyone else find a way of doing that? |
|
My accompanying Twitter thread. Back in 2016/2017, I looked into a local emulation option for service bus as the company I was working at was in a position to need something like it. Today, I find myself in a similar situation again as we don't have the finances to waste, arbitrarily provisioning cloud resources for each developer and need a consistent and repeatable local development experience. I didn't really get into it much back then, but feeling a bit more motivated now, I'm sadly seeing things that are a bit off-putting about how badly the message is being missed. The first result most tend to find is this stackoverflow question which has an answer that's been accepted. The answer points to this feedback forum question from 2012 with at the time of this comment, 149 votes. What galls me about the response is its sheer combined temerity and inaccuracy:
It has been seven years since 2012. The interest on this uservoice is strong enough to warrant further investigation, yet here's a followup with 68 votes of its own. Today, I decided to ask again after taking the time to do a bit of light research, collating the signals actual users and the community have been trying to send to the service bus team. The response?
Similar to a prior poster in this thread, when @clemensv was paged into the discussion by @cmatskas, the response was unhelpful and really, seemed to talk past me. It's a little less great in the prior thread from @PureKrome which even seems to get a bit condescending...
While I disagree with this quip on the basis of my own use case, all that can be considered beside the point - it again talks right past the concerns on the basis of a singular ideal. I'm not sure what the service bus team is waiting for on this one or what would qualify as sufficient. Certainly, a lot has been done to dilute, defer, confuse or even outright dismiss demand for this feature from multiple inquiries fielded seemingly out of hand in multiple forums. From what I'm reading, I think when it comes to a local service bus emulator, it's definitely at a point that warrants action. If it really and truly is the worst idea possible in this context, do us all a favour and put a clear and official explanation in the service bus documentation as to why we shouldn't want a local emulator and what our alternative should be on Azure, specifically for topics. Finally, I also think the responses so far are inconsistent both as they pertain to my perception of a modern-Microsoft, but also what many people value as a first class, competitive option. |
|
"No local dev for anything of serious complexity"... hmm... Kafka can be hosted in a container. Maybe can someone give us a technical explanation why this is not possible, or maybe they want to be so. |
|
A nice little update today...
Sending good thoughts in the hopes that they find a way to make this happen soon! |
|
This is the 3rd time/company have been looking in to this very thing. How to do local development and integration testing with ASB. Every other part of the stack can be run locally in a container, except this. I would be great to get something like Azurite that is API compatible and containerized. |
|
Until then you can try this https://github.com/SwissLife-OSS/squadron |
|
It would be very useful to have this when working offline or as part of our unit testing/integration testing systems. |
|
This just amazes me. I don't know how many times I have been in a disconnected situation (e.g. 14 hour plane flight overseas) where I would love to get some of my development work done but can't because somebody, somewhere decides to take that kind of a stance. Can I work around it or on something besides the ASB part? Sure. But why should I have to be told what I can and can't work on based on my connected status. |
|
It would be really really helpful to have an Azure service bus emulator. |
|
I am not sure I see the point of this unless the main ASB engine is put
into a container so you can use the real thing to test for performance and
behaviour wihout having to deal with the vagaries of the cloud link in the
way.
Just emulating te API doesn't provide much value.
As I understood it originally you were suppossed to be able to run Azure in
your own data center and even run as hybrid so you could use the cloud if
you needed more resources, and there was literature outlining this. If it
was architected in this manner, than re-engineering the ASB services out
into a container should be easy.
Writing a whole emulator that emulates the performance and behaviour of ASB
will be a big and maintenance heavy task otherwise. Extracting out to a
container must be the easier option, certainly in terms of future
maintenance.
If Rabbit can do it why can't Microsoft ? Maybe that question answers
itself...
…On Sat, 20 Jan 2024 at 23:04, Orney ***@***.***> wrote:
Any update here ??
—
Reply to this email directly, view it on GitHub
<#223 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AFCKEYXZ3NTN6D7FBMEP3ITYPREORAVCNFSM4FPYZXLKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJQGIZTINRWGM2A>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
I disagree. In my opinion there's no point in testing for performance on different hardware that what you will run on in production. For example your development machine may be quite beefy compared to the resources you will have in the cloud on a shared infrastructure, so the test you can conduct locally will not be representative of what you will experience in production, even if somehow you will be able to run the whole ASB on your dev box. The point of this emulator is to simplify local development without requiring to create an ASB instance for developer or come up with funky strategies to share the same ASB across multiple developers without getting into each other's way. |
|
I'm pretty sure 99% of people who's looking forward to this just want an easier time during local development and be able to write integration tests that uses the real sdk. |
|
The point is for local testing when you're not using an abstraction. To test the interaction between application and test in an isolated environment. For benchmarking performance, you shouldn't really use something local that you believe mimics performance. Constraints like network latency and bandwidth, even locally, skew those tests. Benchmark your code with your own coded stubs instead. This is aimed at those that want an integrated local development environment so they can run without having to setup cloud environments. For those that want to run an integration test inside a pipeline. It's absolutely useful. I think for your specific usecase, there are better ways to achieve the kind of confidence you're looking for, this will likely not be it. To say that it's pointless though is wrong, and neglects to see who other people's workflows are built. |
|
I'm sticking to my opinion :-)
I'm not talking about benchmarking performance, but being able to test
against a system that has repeatable performance (that I can control if I
want by putting it is a constrained VM, for example).
If it is a local emulator that doesn't have all the bugs and performance
profile of the real ASB then testing against it is not much value - and I
can easily use some other queuing system instead and abstract away the API.
"there are better ways to achieve the kind of confidence you're looking for"
not sure I see what these are, apart from running the same test a large
number of times at different times of the day against the cloud ASB and
hoping that has covered all the bases...
…On Sun, 21 Jan 2024 at 11:00, MartinDotNet ***@***.***> wrote:
The point is for local testing when you're not using an abstraction. To
test the interaction between application and test in an isolated
environment.
For benchmarking performance, you shouldn't really use something local
that you believe mimics performance. Constraints like network latency and
bandwidth, even locally, skew those tests. Benchmark your code with your
own coded stubs instead.
This is aimed at those that want an integrated local development
environment so they can run without having to setup cloud environments. For
those that want to run an integration test inside a pipeline.
It's absolutely useful. I think for your specific usecase, there are
better ways to achieve the kind of confidence you're looking for, this will
likely not be it. To say that it's pointless though is wrong, and neglects
to see who other people's workflows are built.
—
Reply to this email directly, view it on GitHub
<#223 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AFCKEYU27JKWEOIB3LRRKTDYPTYLLAVCNFSM4FPYZXLKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJQGI2TSMRYGEYQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
@AndrewDCDrummond Your use case is different than what people are asking for in this request. If you care about your case, create another request (issue) and ask people to vote there. You shouldn't dismiss the ask here (local emulator for development purposes) as "useless", because many people (myself included) would love to see an emulator as asked here. |
The first part of that is that it may not feel useful, to you. Many other people (based on my experience and following this thread for a long) do not want performance, they want functionality. They don't to add in NServiceBus or MassTransit for the sole purpose of running locally in development scenarios. The second part is that performance testing in a local environment provides little to tell you what would happen in production, against a real ASB, with all the networking considerations. Real performance testing requires isolation and benchmarking of each component you own, then observing a real environment. Using a VM, and constraining CPU is making a lot of assumptions about the real ASB architecture works (I can assure you that the way it runs in the cloud, those aren't the constraints that will allow you to see things you care about). Distributed SaaS/PaaS systems like ASB involve multiple different sub systems running in a highly distributed architecture. They run across multiple AZs and Regions which introduces interesting latency and halts into the production system from initial contact to download/upload times. Boiling that down to a VM that you can control is pretty far off-base in terms of modern performance testing techniques when it comes to cloud services. Testing performance when system interaction is at play comes down to tweaking latency and injecting failures. You want to know what happens when the producing fails, or consumption fails mid-transmit. For the confidence you want, you add a stub in your code that allows you to inject those failures as part of whole service tests, stubbing out calls that go out of your codebase. Then for the rest you need to closely monitor production, and potentially have synthetics for that too. To be clear, this isn't theory, this is how I've done it at massive scale, 100k+ messages a second. I've worked on this with multiple organisations and it's what I do for a job. You can certainly do what you're wanting with an abstraction and something like Rabbit, but if you've ever tried to run Rabbit at scale in the cloud with the same level of redundancy you get with ASB, you'll have an idea of how hard it is to replicate that with a VM locally and a single instance. (This is in relation to the question of how you can do it better, more than happy to consult on the approach) |
|
This would be a good this fit for the .NET Aspire story!
|
Yes, when this is available, we would use this in .NET Aspire - just like we are using the CosmosDB local emulator and Storage emulator. |
|
Local emulators and/or docker instances for cloud-based services are absolutely useful. If this weren't so there wouldn't already exist such tools for storage, CosmosDB, etc. The oddity here isn't that some folks would find this is useful it's that anyone would argue that it isn't and that it wasn't written ages ago... |
|
+1 for adding support for an emulator for local development. Especially now that Aspire supports using emulators, I think it makes sense to include this on some roadmap. |
|
The .net sdk too 🎉 Azure/azure-sdk-for-net#41750 |
|
Just to make sure I have understood - this isn't released yet is it? Just an indication that its coming? |
|
We are currently doing active development on this feature, and expect to have more to share around an initial release by end of the year. Additionally, the local emulator for Event Hubs will go into public preview in the first half of this year. |
|
(on behalf of many long suffering devs, here ....) |
|
Hello Everyone, We’re excited to share that the much-anticipated Event Hubs emulator is now live! The first version was launched today. You can find more details here. Our team has put in a tremendous amount of effort over the past few months to make this happen, and we’re eager to hear your thoughts on it. That being said, we’re still working on the emulator for Service Bus. We currently expect to have an initial version ready by the end of the year. We deeply appreciate your patience and eagerly await your feedback! |
|
the links in your announcement are broken.... might wanna recheck them in incognito |
Links should be fixed now, kindly retry and let me know if you still face any issues. @razvangoga |
|
When the event-hub with Kafka protocol support? |
|
@Saglodha is there any place we can follow up on some updates or you will update here? It is really long awaited piece of software, so we would be happy to know if we can expect some sort of alpha/beta testing etc soon? |
@jabrena - We're currently working on introducing Kafka support in Event Hubs emulator by end of CY24. I would keep this forum updated with future updates. |
@murbanowicz - Thank you for your patience. I would continue to share future updates in this forum. Our team has been diligently working on Service Bus emulator, and we are aiming for a release by the end of CY24. |
|
Hey folks, I thought this would be a fun side project so I made LocalSandbox, a free and open source azure emulator that supports azure service bus. It currently supports sessions, dead-lettering, subqueues, scheduled messages, and more. I've tested it with both the official node.js SDK and the official c# service bus SDK. Would love to get thoughts on it! I'm also open to creating emulators for more services if there's interest. |
and others
Trying to get an understanding of what the local development story is. RabbitMQ for example can be installed on a dev machine, or run as a Docker container. If I have development/build images that need Azure Service Bus, what's the story of getting a runnable instance locally?
The text was updated successfully, but these errors were encountered: