Not yet, but @U06QSF3BK and @U0510KXTU will do some research. Maybe we will be able to run just Clojure on the JVM without GraalVM 😀

seems like it :)

I did try it, and it does give a nice speedup: https://clojurians.slack.com/archives/C01UQJ4JC9Y/p1669711630530119?thread_ts=1669707352.165539&cid=C01UQJ4JC9Y

but there are some caveats, like that the snapshots are cached for 14 days

500ms, we can do better :P

yeah well that is me 😛

Cached for 14 days if you do not use it.

oh, actually, 500ms is quite good isn't it?

sure can be better

yeah I think so

It depends. What architecture?

like, from 7s -> 5ms

x86 only supported now for snapshots I think

it relies on Firecracker VM snapshots

https://github.com/firecracker-microvm/firecracker/blob/main/docs/snapshotting/snapshot-support.md

How it compares with HL GraalVM?

270ms cold start with HL GraalVM

in the future, the JVM will probably have some image capabilities too, as project Leyden, but this will take a while

yeah Leyden interesting, I think there are broad fronts into this

Seems HL native backend still wins a little bit :D

I think we don't need to throw native-image totally away, this is just something different

like, one could do preloading stuff into the snapshot

well, ok, snapshot now only supported for jvm

and with bb holy lambda you can still edit code in the console ;)

Btw, what is the project Leyden?

but if it would be supported for other runtimes, was thinking that one could use native-image and the do some computation that would be preloaded, like NLP model loading etc.

https://openjdk.org/projects/leyden/

btw @U06QSF3BK it was nice to meet you at the Dutch Clojure Days. Your nREPL that went into nbb now also lives in #C03DPCLCV9N

it was nice meeting you too 🙂

I think post outlines Leyden better: https://openjdk.org/projects/leyden/notes/02-shift-and-constrain with > https://openjdk.org/projects/leyden/notes/02-shift-and-constrain#roadmap > We have two categories of work before us: Specify and implement the concept of condensers, and research and develop specific condensers and related new language features. there was an earlier post too https://mail.openjdk.org/pipermail/discuss/2020-April/005429.html, which sounded more like re-awakening jaotc/jlink, but I guess the idea is to allow these features more broadly in the java language

I think I should also try joyride/VSCode, fearing that it might bump be away from Emacs, let's see 😄

Well, the previous description depicts Leyden as some kind of GraalVM native-image competitor?

not competitor, native-image is one of the many possible approaches, Leyden is more broader and more configurable take and native-image fits into that specification (or so)

Leyden sounds ... complex ... ! Wonder how the dev-ex will be! I also cannot quite gather from the post if it's relevant for other JVM languages or java-only?

thinking that Leyden is about moving the faster startup time underpinnings into the Java Platform Specifications, so I guess other languages that target the JVM could generate bytecode that is more suitable for these kinds of optimizations (not necessarily new type of bytecode)

In my systems cold start includes init for vpc ens and other infra so anything less than 1 second is excellent if the Dev ex is also good

@U04V15CAJ I have tried it now. you can see my interim results here https://clojurians.slack.com/archives/C01UQJ4JC9Y/p1670564880315559?thread_ts=1670372673.558019&cid=C01UQJ4JC9Y

very nice!

it really is. just setting up my devops scripts for it using bb/cli now

p.s. shame about the penalties

penalties?

oh you mean soccer, I'm blissfully ignorant of all of that ;)

today your ignorance is providing extra bliss (c.f. with my family)

Unofficial list https://gist.github.com/shortjared/4c1e3fe52bdfa47522cfe5b41e5d6f22. The aws docs suggest you can find some by digging through the list of services that work with IAM. Not very helpful. > The identifier for a service principal includes the service name, and is usually in the following format: > _service-name_. > The service principal is defined by the service. You can find the service principal for some services by opening https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_aws-services-that-work-with-iam.html, checking whether the service has Yes in the Service-linked role column, and opening the Yes link to view the service-linked role documentation for that service. Find the Service-Linked Role Permissions section for that service to view the service principal. > The following example shows a policy that can be attached to a service role. The policy enables two services, Amazon ECS and Elastic Load Balancing, to assume the role. The services can then perform any tasks granted by the permissions policy assigned to the role (not shown). To specify multiple service principals, you do not specify two Service elements; you can have only one. Instead, you use an array of multiple service principals as the value of a single Service element.

Ty leaf

Not sure if you're misunderstanding what the writer is? A writer does both reads and writes - so if your service needs to be able to write, it needs to use the writer. It'll have read-access as well through that endpoint though. Using the reader is not necessary, but it can be a good idea for security reasons, if a specific services only needs read-access (eg a BI application), and for performance as well, if you want to save some load on the writer.

My thought was, that AWS would have a kind of “load balancer” connection string: It would take SQL commands, and if it was an “write”, send it to the writer instance. If it was a read, it would balance the requests to both writer and reader. But indeed, I realize this is not how things work xD. I guess I would just have two connection pools, and write in my application which queries I want to get where

The reader is not necessarily in sync with the writer - so that would not be consistent

From https://aws.amazon.com/rds/aurora/faqs/ > How far behind the primary will my replicas be? > Since Amazon Aurora Replicas share the same data volume as the primary instance in the same AWS Region, there is virtually no replication lag. We typically observe lag times in the tens of milliseconds

> I guess I would just have two connection pools, and write in my application which queries I want to get where Yeah, exactly. If you can sacrifice some consistency, then you can use the reader in some cases - but that would be very application-specific.

Makes sense, thanks @U052XLL3A!

I'm pretty sure that by default, if you create an ec2 instance and use aurora within the same aws region, AWS will be smart about the routing, and it'll just work -- by default. But networking config can be customized in all sorts of ways in AWS, and be arbitrarily complex, so I'm not sure how to verify that a specific configuration isn't problematic in some way. I have always had trouble finding a good to-the-point guide about networking concepts for AWS. If anyone has a good link, it'd be much appreciated!

One thing that you can do is to log on to the ec2 instance and then do a `nslookup` traceroute to the aurora host and see what happens

I spoke with an aws solutions architect — they say that if the db is publicly accessible, it will use the public internet, but if it’s set to vpc only, it will use the private internet. I was surprised that the network layer wouldn’t be smart about this. I’ll look deeper into traceroute!

Could be interesting to verify it 👍 I'm pretty sure it depends on the routers involved, and it sounds odd that they wouldn't optimize for the least amount of hops no matter what url is used. But I guess there's some reasoning behind it ...

When I run traceroute I see:

traceroute to ***. (), 30 hops max, 60 byte packets
 1  * * *
 2  * * *
 3  * * *
 4  * * *
 5  * * *
 6  * * *
 7  * * *
 8  * * *
 9  * * *
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *

If I do a traceroute from my local machine to a cloudfront-hosted website, I get ~15 hops, some of which go past my ISP and then end up at cloudfront.

I don't have an aurora db or ec2 instance at hand so I can't do a similar test.

But 30 hops sound like alot, if it's from one aws host to another

what happens if you put the ec2 and aurora inside the same vpc and repeat the experiment?

btw I also get a few lines with * - maybe some routers don't want to reveal themselves

Just tested — I get the same thing! Iinteresting

(I disabled public access and ran traceroute, and got stars all the way down to 30)

do you have access to the aurora instance at all from the ec2? eg can you do telnet [host] [port] and get a reply back?

(aurora instance is probably the wrong term 😉 )

the ec2 instance doesn’t have telnet, but I can confirm I have access; I was able to set up a jdbc connection pool from a clojure repl that I set up on it

I think you can install telnet with sudo yum -y install telnet, but yeah, sounds like things work. not sure why the traceroute isn't helpful

Update: here’s what makes me think that it does indeed use the vpc. If I write nslookup inside ec2, I get an ip address that’s like 172.xx But if I nslookup from my mac, it’s a more “external looking” ip address.

https://repost.aws/questions/QU0d7EgUotRfWuExYl4b-0Nw/how-does-ec-2-hop-to-a-publicly-accessible-rds-endpoint <- on aws too

And does it then return the external ip if you nslookup from ec2 without the vpc being active?

Ooh good question. I am not quite sure sure how I would disable the vpc. I only have one (the default), would this break something? I’ll note to look into it!

It indeed uses the internal network. As you observed, the IP address associated with that hostname is a private IP address. Such addresses aren't routable on the internet, only within a private network. You can also see this with host <dns name> (try host -v or dig and you'll see more details) For other services like S3, that are not deployed within VPC, there are "VPC endpoints" to make sure your traffic uses a dedicated link within Aws network and doesn't go through the Internet.

This is for RDS but I guess that's the same for Aurora? Can't find the private IP address for my Amazon-RDS instance https://serverfault.com/questions/601548/cant-find-the-private-ip-address-for-my-amazon-rds-instance > RDS instances can change their IPs unexpectedly, so they should not be used nor are they provided in the console or API (although you can technically dig for them). The DNS endpoint provided in the AWS console will resolve to the internal IPs from within Amazon's network.

You can also try reachability analyzer: https://docs.aws.amazon.com/network-manager/latest/tgwnm/route-analyzer.html It is a bit complicated (with rds at least) but these help • https://repost.aws/questions/QUyE7QG44dQtaUNe6unO-KPg/possible-to-run-aws-vpc-reachability-analyzer-from-lambda-eni-to-rds-endpoint • https://serverfault.com/questions/601548/cant-find-the-private-ip-address-for-my-amazon-rds-instance ◦ see filtering network interfaces by security group