This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-09-29
Channels
- # announcements (1)
- # babashka (120)
- # beginners (184)
- # cider (14)
- # clara (2)
- # clj-kondo (25)
- # cljfx (9)
- # cljsrn (43)
- # clojure (46)
- # clojure-australia (2)
- # clojure-berlin (5)
- # clojure-dev (2)
- # clojure-europe (10)
- # clojure-france (3)
- # clojure-nl (1)
- # clojure-spec (14)
- # clojure-uk (30)
- # clojurescript (50)
- # code-reviews (19)
- # conjure (11)
- # core-logic (2)
- # cursive (20)
- # datascript (1)
- # datomic (74)
- # figwheel-main (1)
- # fulcro (10)
- # funcool (2)
- # graphql (1)
- # lambdaisland (1)
- # malli (2)
- # meander (22)
- # nrepl (13)
- # off-topic (28)
- # overtone (3)
- # pathom (7)
- # pedestal (4)
- # re-frame (4)
- # reagent (16)
- # reitit (4)
- # releases (1)
- # ring (8)
- # shadow-cljs (93)
- # specter (6)
- # sql (13)
- # test-check (1)
- # tools-deps (1)
- # tree-sitter (2)
- # vim (8)
- # xtdb (25)
Hi. Is there any way to make custom transaction function to omit operaton (e.g. return nil instead of transaction operation)? I want omit insertion of data in some situations.
I have permission entity with composite tuple (unique) on all three attributes. When I try to bulk insert list of permissions transation sometimes aborts with unique exception.
I want to make something like postgres's on conflict do nothing
. Here is my transaction function, which not workig obviously.
(defn try-add-permission
[db {:keys [permission/app
permission/user
permission/role] :as perm}]
(if (d/q '[:find ?p .
:in $ ?app ?user ?role
:where
[?p :permission/app ?app]
[?p :permission/user ?user]
[?p :permission/role ?role]]
db app user role)
nil
perm))
I already tried that. Got error
{:status :failed,
:val #error{:cause "Cannot write auth.server.cas_sync$try_add_permission@723b2a26 as tag null",
:via [{:type java.util.concurrent.ExecutionException,
:message "java.lang.IllegalArgumentException: Cannot write auth.server.cas_sync$try_add_permission@723b2a26 as tag null",
:at [datomic.promise$throw_executionexception_if_throwable invokeStatic "promise.clj" 10]}
{:type java.lang.IllegalArgumentException,
:message "Cannot write auth.server.cas_sync$try_add_permission@723b2a26 as tag null",
:at [org.fressian.handlers.WriteHandlerLookup
requireWriteHandler
"WriteHandlerLookup.java"
48]}],
...
quoting does not help. And docs does not use quoting https://docs.datomic.com/on-prem/database-functions.html#using-transaction-functions
This is a quoting issue. The exception is related to serializing the function, which you can’t do. Your function is not being executed yet
In fact your transaction data hasn’t left the peer. What is the error you get when you quote the function name?
I get error: Could not locate auth/server/cas_sync__init.class, auth/server/cas_sync.clj or auth/server/cas_sync.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name.
you’ll note in your link you need to make classpath transaction functions available on the classpath of the transactor. This error looks like it can’t find the function
@U09R86PA4 thanks, It seems that I misunderstood how transaction functions work.
Updating datomic cloud compute group to 2020/09/23 715-8973 release, the log shows that the compute nodes don’t seem to get up after upgrade… complaining that our application code has syntax error (which it shouldn’t as it worked in previous version)
yes, workaround with support… it seemd you can’t have paths that point to for example `“../common/src” (like we have sharing backend and frontend cljc code)
When I connect a web ion via an API Gateway proxied thru a lambda, my ion function is supposed to receive a ring-compatible map as an argument (according to the official Ion docs). However, the map I receive only contains :headers, :server-name and a :datomic.ion.edn.api-gateway/data and /json keys, so I can't just use the typical routing libs to build my web-app or http API, because those depend on the :request-method and :uri keys of the request map. Is it a know issue? Is it something related to the Lambda proxy data format version? Is it just some kind of mis-configuration?
here is an example request map I observed:
{:headers
{"accept-encoding" "gzip, deflate",
"content-length" "0",
"host" "",
"user-agent" "http-kit/2.0",
"x-amzn-trace-id" "Root=1-5f730b24-4ac64db84deabaf53c38af60",
"x-forwarded-for" "42.200.88.157",
"x-forwarded-port" "443",
"x-forwarded-proto" "https"},
:server-name "",
:datomic.ion.edn.api-gateway/json
"{\"version\":\"2.0\",\"routeKey\":\"$default\",\"rawPath\":\"/\",\"rawQueryString\":\"\",\"headers\":{\"accept-encoding\":\"gzip, deflate\",\"content-length\":\"0\",\"host\":\"\",\"user-agent\":\"http-kit/2.0\",\"x-amzn-trace-id\":\"Root=1-5f730b24-4ac64db84deabaf53c38af60\",\"x-forwarded-for\":\"42.200.88.157\",\"x-forwarded-port\":\"443\",\"x-forwarded-proto\":\"https\"},\"requestContext\":{\"accountId\":\"191560372108\",\"apiId\":\"8g759uq7nb\",\"domainName\":\"\",\"domainPrefix\":\"8g759uq7nb\",\"http\":{\"method\":\"GET\",\"path\":\"/\",\"protocol\":\"HTTP/1.1\",\"sourceIp\":\"42.200.88.157\",\"userAgent\":\"http-kit/2.0\"},\"requestId\":\"Tn6trha8yQ0EMGg=\",\"routeKey\":\"$default\",\"stage\":\"$default\",\"time\":\"29/Sep/2020:10:23:32 +0000\",\"timeEpoch\":1601375012276},\"isBase64Encoded\":false}",
:datomic.ion.edn.api-gateway/data
{:version "2.0",
:routeKey "$default",
:rawPath "/",
:rawQueryString "",
:headers
{:accept-encoding "gzip, deflate",
:content-length "0",
:host "",
:user-agent "http-kit/2.0",
:x-amzn-trace-id "Root=1-5f730b24-4ac64db84deabaf53c38af60",
:x-forwarded-for "42.200.88.157",
:x-forwarded-port "443",
:x-forwarded-proto "https"},
:requestContext
{:routeKey "$default",
:stage "$default",
:time "29/Sep/2020:10:23:32 +0000",
:domainPrefix "8g759uq7nb",
:requestId "Tn6trha8yQ0EMGg=",
:domainName "",
:http
{:method "GET",
:path "/datomic",
:protocol "HTTP/1.1",
:sourceIp "42.200.88.157",
:userAgent "http-kit/2.0"},
:accountId "191560372108",
:apiId "8g759uq7nb",
:timeEpoch 1601375012276},
:isBase64Encoded false},
}
my ion-config.edn looks like this:
{:allow [datomic.ion.starter.http/ionized-app]
:lambdas {:app
{:fn datomic.ion.starter.http/ionized-app
:integration :api-gateway/proxy
:description "return html app"}}
;:http-direct {:handler-fn datomic.ion.starter.http/return-something-json}
:app-name "kyt-dev"}
I'm using the Solo topology (the version, which was the latest last week), otherwise I wouldn't bother with lamdba gateways if I could use the production topology.
the docs are mentioning these /json and /data keys in a note, but just in the table above the note, they are not namespaced: https://docs.datomic.com/cloud/ions/ions-reference.html#web-ion
Hey @U086D6TBN , have a look at https://github.com/pedestal/pedestal.ions And https://github.com/pedestal/pedestal-ions-sample
thanks, I had a look, but I don't see how would it deal with my situation.
it does have a great example of a protocol which converts the response body into an input stream, which I still need, because the reitit.ring/create-resource-handler
just returns a java.io.File as a :body and Datomic threw some ->>bbuff conversion error as a result.
for now, I just have a middleware to transform the above mentioned request map to be ring compatible:
(if-let [gw-data (:datomic.ion.edn.api-gateway/data gw-req)]
(-> gw-req
(assoc :uri (-> gw-data :requestContext :http :path))
(assoc :request-method (-> gw-data :requestContext :http :method)))
gw-req)
I want to make sure I understand, did you call apigw/ionize
on your ring handler function?
https://docs.datomic.com/cloud/ions/ions-tutorial.html#lambda-proxy
I have the suspicion that or ion-config.edn doesn't need the :integration :api-gateway/proxy
option anymore if I use the newer style HTTP API gateway setup (as opposed to the RESTful API style), it's just hasn't been documented...
@U0CJ19XAM yes, that .../ionized-app
is defined as (apigw/ionize app)
, where app is simply:
(fn [req]
{:status 200
:headers {"content-type" "text/plain"}
:body (with-out-str
(clojure.pprint/pprint req))})
To make sure I understand correctly, you're not using the supported integration. Is there still a problem if you use the supported one?
I'm just realizing that probably the Datomic docs are taking about how to integrate a web ion with the traditional RESTful API gateway, not the new "HTTP API". I'm using this "new style" gateway, because it supports JWT authorizer's out of the box, without the need to deploy a lamdba function just for that purpose.
yes, the mentioned request map was observed when my ion-config.edn contained that : integration :api-gateway/proxy
setting
my API gw was created my this sample CF template though: https://github.com/awsdocs/amazon-api-gateway-developer-guide/blob/master/cloudformation-templates/HTTP/http-with-jwt-auth.yaml which I found in these AWS docs: https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-jwt-authorizer.html
I'm not sure that approach is compatible with the current mechanism for making a ring compatible handler. I think you're in uncharted territory, rife with undefined behavior.
I will probably just transition to the production topology, though I can foresee issues with the VPC link and NLB in that case, which I have just as little experience with as I have with Cognito and JWT authorizers :)
Alternatively, I can write a replacement ionizer function for this new "HTTP API" gateway
You're free to do that, but we won't be able to support that if there are issues.
@U086D6TBN try switching the payload format version in the integration config for your API Gateway instance. Saying that based purely on memory so details may be off, but I’m pretty sure I got cut by this exact issue and that was the solution that I eventually found.
@U056QFNM5 thanks a lot for the advice. it worked indeed and I can even see how the JWT authorizer has decoded the token!
You're very welcome. Also keep an eye out for setting cookies. I ran into an issue where the value of my set cookie header in my ring response map was a vector rather than a string. Apparently this is legal in ring, but it didn't work in an Ions context. Again, going from memory here, but I think that was right. A simple middleware to detect such values and only take the first value out of the vector worked.
Hi guys, just found this thread because I'm working on the exact same thing right now (trying to deploy an SPA as ion / lambda proxy) I got confused by the mismatch between the Ion tutorial and the API Gateway console, so just to clarify once more: What the Datomic Ion docs are talking about is now called REST API on AWS? And the HTTP API is a new thing, that is not officially supported?
Yes, REST is the old kind that the docs implicitly refer to. HTTP can work, but it's not what the docs describe specifically.
Just gotten bitten for an hour getting 401 Unauthorized for Datomic Pro due to missing XML schema in ~/.m2/settings.xml
, which is not mentioned in https://my.datomic.com/account. Previously: https://clojurians-log.clojureverse.org/datomic/2019-01-30/1548890962.888000
Sorry about that, what is missing in https://my.datomic.com/account ? I see the .m2/settings.xml described as:
;; In ~/.m2/settings.xml:
<!-- ~/.m2/settings.xml (see the Maven server settings docs) -->
<servers>
…
<server>
<id></id>
<username>REDACTED</username>
<password>REDACTED</password>
</server>
…
</servers>
;; In deps.edn:
{:mvn/repos
{"" {:url ""}}
:deps
{com.datomic/datomic-pro {:mvn/version "${VERSION}"}}}
<?xml version="1.0" encoding="UTF-8"?>
<settings xmlns=""
xmlns:xsi=""
ssi:schemaLocation="">
ack! let me talk to Alex so i fully understand and I can update our http://my.datomic.com account to reflect that!
Also - and this is probably out of scope - but even after fixing settings.xml (and running clj from terminal), IntelliJ Cursive still reported a 401 for deps because it caches the 401 error for a given deps.edn (not sure if this is due to Maven or IntelliJ). Fixed after reordering any two items under :deps, then I could start a REPL.
I'm now debugging this 401 issue in my own case, where Datomic fails to download consistently on Github Actions for CI purposes but not locally on my machine.
Hey! I have a transaction that always puts one entry into datomic, and I'd like to get it to return the entity id of the new entry. I notice that the returned map contains :tx-data, which has the data I need. But I'm not sure how to read the contents of the returned datum, and, indeed, if this is considered bad best practices or not. Help appreciated!
I'm getting an empty map on that one, do I just need to add a temp-id to the transaction?
if your transaction included tempids, datomic returns the resolved ids after it transacts
{:tx-data [#:user{:email "e", :password "q", :name "q", :token "q"}]} ; this query is generated by (make-record :user) and executed
(def q (make-record :user "q" "q" "q" "q"))
(:tempids q)
Ah, ok! Just had to add :db/id "nonsense"
to my query and now the map gives me {"nonsense" id} !
thanks!
if :user/email
is a unique attribute in your database, you can use it to lookup entities without entity ids
Oh, nice