Fork me on GitHub
#datomic
<
2018-12-05
>
hansw11:12:07

I have a query that looks like this:

[:find (pull ?e [*])
            :in $ [?veh ...]
            :where
            [?e :vehicle/plate ?veh]]
This returns one value, where multiple rows are squashed together into one map, which I don't understand.

souenzzo11:12:54

:find [(pull ?e [*]) ...] works?

hansw11:12:36

but it's squashing multiple rows into one map

hansw11:12:24

so the result shape is [[{}]]

hansw11:12:38

where there are multiple rows inside the {}

hansw11:12:41

eg.

{:vehicle/a "foo" :vehicle/b "bar" :vehicle/a "more" :vehicle/b "other"}

hansw11:12:40

so I can't distinguish 'rows' from each other

hansw11:12:14

What I need is:

[{:vehicle/a "foo" :vehicle/b},
 {:vehicle/a "more" :vehicle/b "other"}
]

hansw11:12:17

i think it broke after upgrading my datomic client...

souenzzo11:12:41

well. let's wait someone from datomic team 🙂

hansw13:12:44

maybe there were subtle changes when the client-api changed from datomic.client to datomic.client.api

hansw13:12:56

in any case the q function changed from requiring a conn parameter

hansw13:12:01

to just the argmap

thegeez13:12:28

Could you paste the output you are seeing? Your first example is not valid clojure with the same :vehicle/a key appearing multiple times in the same map

hansw13:12:16

i made it shorter for brevity sake

hansw13:12:47

one moment

hansw13:12:15

ooh wait yes

hansw14:12:51

it seems that i was in desperate need of coffee when i pasted this

hansw14:12:43

ok, this was another case of confessional debugging.

thegeez14:12:11

Remote rubber ducking works 🙂

👍 8
hansw14:12:13

thnx for listening 🙂

PB16:12:53

Hey there. Last night I was trying to stand up a pedestal service on IONS. I have been following: https://github.com/pedestal/pedestal-ions-sample However, when trying to deploy, as such:

clojure -Adev -m datomic.ion.dev '{:op :deploy, :group "ion-pet-Compute-1Q6752A2P837M", :uname "pet-service-sample"}' 

{:execution-arn
 arn:aws:states:us-east-2:272695641059:execution:datomic-ion-pet-Compute-1Q6752A2P837M:ion-pet-Compute-1Q6752A2P837M-pet-service-sample-1543987389875,
 :status-command
 "clojure -Adev -m datomic.ion.dev '{:op :deploy-status, :execution-arn arn:aws:states:us-east-2:272695641059:execution:datomic-ion-pet-Compute-1Q6752A2P837M:ion-pet-Compute-1Q6752A2P837M-pet-service-sample-1543987389875}'",
 :doc
 "To check the status of your deployment, issue the :status-command."}
I get the following
clojure -Adev -m datomic.ion.dev '{:op :deploy-status, :execution-arn arn:aws:states:us-east-2:272695641059:execution:datomic-ion-pet-Compute-1Q6752A2P837M:ion-pet-Compute-1Q6752A2P837M-pet-service-sample-1543987389875}'

{:deploy-status "FAILED", :code-deploy-status "FAILED"}
Looking at the logs it's clear that I have given the wrong :deployment-group:
2018-12-05T05:23:20.008Z    059ddc53-7801-4f4c-b688-60918314b781    DeploymentGroupDoesNotExistException: No Deployment Group found for name: ion-pet-Compute-1Q6752A2P837M
at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/json.js:48:27)
at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:683:14)
at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)
at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)
at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10
at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)
at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:685:12)
at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
My question is where do I find that? I copied it direclty from the cloudformation name. Is that incorrect?

Joe Lane16:12:23

What version of clojure are you using?

Joe Lane16:12:32

Also, have you deployed the datomic cloud system correctly, but can you confirm it by connecting to that system before this push?

PB16:12:27

I believe I have deployed it correctly. I verified that by testing following the getting started docs

Joe Lane16:12:28

When I deploy something I dont have quotes around the group. clojure -Adev -m datomic.ion.dev '{:op :deploy, :group myproject-dev-compute, :rev "2fee444890ccf58d4629294f3904bf1c38bb762q"}'

Joe Lane16:12:01

I’ve gotta run but I hope thats helpful.

PB16:12:57

Thanks for the input. My group needs to have the random identifier as such or it can't find the group as such ion-pet-Compute-1Q6752A2P837M or it will not do anything

PB16:12:17

I still get the error: 2018-12-05T05:23:20.008Z 059ddc53-7801-4f4c-b688-60918314b781 DeploymentGroupDoesNotExistException: No Deployment Group found for name: ion-pet-Compute-1Q6752A2P837M

PB16:12:51

Can anyone tell me how to find my deployment group. THe docs are pretty bad around this. https://docs.datomic.com/cloud/ions/ions-reference.html#deploy

PB16:12:34

Being that there is no :deployment-group I used $(SystemName)-Compute-$(GeneratedId) but the docs suggest that that is for group

jaret17:12:45

@petr to deploy you replace $(GROUP) with the name of your compute stack. are you sure the C is capitalized in “Compute”? You can see the name in the CF console https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks?filter=active

PB17:12:38

The C is defintiely capitalized. I copied the name of the stack from the UI in cloudfront

PB17:12:21

This is my last failed attempt:

clojure -Adev -m datomic.ion.dev '{:op :deploy, :rev "674e654d429ebc5092be8008b1463617720a1a7c", :uname "pet-service-sample", :group "ion-pet-Compute-1Q6752A2P837M", :region "us-east-2"}'

PB17:12:06

If i omit the random identifier, it doesn't even accept the request

jaret17:12:42

@petr what is your output from your Push command?

jaret17:12:12

it will have a :deploy-command that is generated for you

PB17:12:41

clojure -A:dev -m datomic.ion.dev '{:op :push :region "us-east-2" :uname "pet-service-sample"}'

Downloading: com/datomic/java-io/0.1.11/java-io-0.1.11.pom from 
(cognitect.s3-libs.s3/upload "datomic-code-8fe8a54a-0daf-48b9-a4b8-bbdf996b81ae" [{:local-zip "target/datomic/apps/ion-pet-service/unrepro/pet-service-sample.zip", :s3-zip "datomic/apps/ion-pet-service/unrepro/pet-service-sample.zip"}] {:op :push, :profile "devthenet", :region "us-east-2", :uname "pet-service-sample"})
{:uname "pet-service-sample",
 :region "us-east-2",
 :deploy-groups (),
 :dependency-conflicts
 {:deps
  {commons-codec/commons-codec #:mvn{:version "1.10"},
   org.clojure/tools.analyzer.jvm #:mvn{:version "0.7.0"},
   com.fasterxml.jackson.core/jackson-core #:mvn{:version "2.9.5"},
   org.clojure/tools.reader #:mvn{:version "1.0.0-beta4"},
   org.clojure/core.async #:mvn{:version "0.3.442"}},
  :doc
  "The :push operation overrode these dependencies to match versions already running in Datomic Cloud. To test locally, add these explicit deps to your deps.edn."},
 :deploy-command
 "clojure -Adev -m datomic.ion.dev '{:op :deploy, :group <group>, :uname \"pet-service-sample\", :region \"us-east-2\"}'",
 :doc
 "To deploy, issue the :deploy-command, replacing <group> with a group from :deploy-groups"}

PB17:12:18

clojure -Adev -m datomic.ion.dev '{:op :deploy, :rev "674e654d429ebc5092be8008b1463617720a1a7c", :uname "pet-service-sample", :group "ion-pet-Compute-1Q6752A2P837M", :region "us-east-2"}'
Is what I got when I took the deploy-command and added the group

jaret17:12:24

whats in your ion-config.edn?

jaret17:12:06

Also, @petr what version of Cloud are you running?

PB17:12:20

{:allow    [ion-sample.ion/app]
 :lambdas  {:app {:fn ion-sample.ion/app :description "Exploring Ions with Pedestal"}}
 :app-name "ion-pet"}

PB17:12:29

I deployed it yesterday, so the latest version

PB17:12:47

So I realised that I hadn't committed that change. So I committed pushed and deployed. It's still failing, but this time cloudwatch doesnt' give anything useful


17:31:29
START RequestId: cdd511ba-86b4-4242-9abb-e587c536b404 Version: $LATEST

17:31:29
2018-12-05T17:31:29.248Z	cdd511ba-86b4-4242-9abb-e587c536b404	{ event: { codeDeploy: { deployment: [Object] }, lambda: { cI: 0, c: [Object], uI: -1, u: [], dI: -1, d: [], common: [Object] } } }

17:31:30
END RequestId: cdd511ba-86b4-4242-9abb-e587c536b404

17:31:30
REPORT RequestId: cdd511ba-86b4-4242-9abb-e587c536b404	Duration: 1271.95 ms	Billed Duration: 1300 ms Memory Size: 128 MB	Max Memory Used: 38 MB
No newer events found at the moment. Retry.

PB17:12:35

clojure -Adev -m datomic.ion.dev '{:op :deploy-status, :execution-arn arn:aws:states:us-east-2:272695641059:execution:datomic-ion-pet-Compute-1Q6752A2P837M:ion-pet-Compute-1Q6752A2P837M-pet-service-sample-1544031395342, :region "us-east-2"}'  

{:deploy-status "FAILED", :code-deploy-status "FAILED"}

marshall17:12:38

@petr can you try putting in the region explicitly

marshall17:12:43

as a :region key

marshall17:12:48

in the deploy map

marshall17:12:59

oh, you did

jaret17:12:12

I think you’re on the right track though. This has to be a permissions/creds or region thing. I just tested on a new stack and my deploy command was populated with the compute stack as the $group

marshall17:12:32

well, actually, did you provide an “application name” in your CFT when you launched?

marshall17:12:44

if you did, the group name is that not your comput group name

PB17:12:46

I am an admin on this account. I did provide an application name

PB17:12:00

ion-pet

marshall17:12:05

look in the outputs of the stack in the CF dashboard

marshall17:12:28

sorry, by that i mean the app name

PB17:12:11

SystemName ion-pet System Name

marshall17:12:13

can you look in your cloufformation dashboard

marshall17:12:25

find the stack you launched (the compute stack)

marshall17:12:29

and in outputs find CodeDeployDeploymentGroup

PB17:12:37

CodeDeployDeploymentGroup ion-pet-Compute-1Q6752A2P837M CodeDeploy Deployment Group

jaret17:12:37

what do you have under “AvailabilityZone1”?

PB17:12:49

AvailabilityZone1 us-east-2b AvailabilityZone1

marshall17:12:59

you have a :rev and a :uname -> i dont think that would cause this, but i would expect you only to have one or the other

PB17:12:48

The example I was following specified one:

To push the project to your Datomic Cloud environment, execute the
following command from the root directory of the sample project:

`clojure -A:dev -m datomic.ion.dev '{:op :push :uname "pet-service-sample"}'`

We provide a `:uname` key because the sample has a `:local/root` dependency.

This command will return a map containing the key
`:deploy-command`. Copy the value and execute it at the command line
to deploy the ionized app. You will need to unescape the `:uname` value.

marshall17:12:11

ah right, the local dep

marshall17:12:38

you should be able to look at your list of codedeploy groups

PB17:12:22

I can see 2 attempted deployments

PB17:12:50

Could it be that the example code does not work and its' failing because it's unable to start the app?

marshall17:12:02

not if the error is about not finding the group

marshall17:12:07

if that’s not the error, then yes

marshall17:12:42

if you click on the latest failed deployment you can look at the reported cause

PB17:12:03

So once I made i commited and pushed the change to ion-config.edn, the deployment group error went away

PB17:12:21

But now it just tells me that the deployment failed

jaret17:12:52

yeah, then that means the code is failing/unable to start the app

PB17:12:04

Oh man, that sucks

jaret17:12:24

But hey! We found the deployment group 🙂

jaret17:12:35

It was right where you said it was 🙂

PB17:12:41

Yeah! Thanks for that!

PB17:12:01

So do you think it's that https://github.com/pedestal/pedestal-ions-sample is just not compatible with the new cloud or there is just a mistake somewhere?

marshall17:12:36

Do you see an error in your cloudwatch logs

PB17:12:56

I do not

PB17:12:26

At least not in the compute

marshall18:12:38

latest log group? the redeploy usually creates a new log group within the stream

jaret18:12:26

@petr these docs show an example of navigating the log group for exceptions

PB18:12:46

Thanks!

PB18:12:41

I think I'm going to try to redeploy with the exact name of the system used in the example (later today). I'll report back

lwhorton19:12:51

am i right in thinking that not null requirements on a standard sql column is a recipe for never being able to evolve your schema into the future? if i had foo varchar not null and a year later we deprecate foo to instead use bar, i dont want my application layer to have to carry the baggage of “filling in” old foos as well as bars?

lwhorton19:12:07

i just want to leave the app code alone which deals with foos, and everything new uses bars instead.

eraserhd19:12:14

Postgres, at least, allows efficient dropping of a "not null" constraint.

lwhorton20:12:20

is it more useful to have nothing marked not null than* to pick and choose and in the future drop it?

lwhorton20:12:36

or am i being silly by leaving everything nilable?

eraserhd20:12:43

It depends?

eraserhd20:12:25

In practice, I've seen a lot of Java code that has to check if every thing it touches is null. This comes from not having a coherent idea throughout the system of what null means (error? not yet filled in?). This is wasteful and super painful. The "not null" constraint on fields might be useful to communicate to other devs, who cannot push code which violates contracts. That said, there are other ways to validate contracts, and which is best depends on the domain.

eraserhd20:12:56

And if you don't have other developers, it's moot. Unless you have an old, dodgy brain like me.

lwhorton20:12:28

haha, :thumbsup:

Alex Miller (Clojure team)20:12:21

Datomic (and Clojure) strongly embrace the idea that you should say things that you know and omit things that you don’t

Alex Miller (Clojure team)20:12:42

or omit stating the absence of a thing, if that makes sense

4
Alex Miller (Clojure team)20:12:05

so, I do not think it is silly

lwhorton20:12:27

i just watched rich’s last talk “maybe not” and was trying to apply it to this case where i dont have datomic but want datomic in postgres

Alex Miller (Clojure team)20:12:29

but as with anything, it depends :)

lwhorton20:12:24

i do like the idea of nil means ‘i dont know’ not ‘empty to satisfy a constraint’

Alex Miller (Clojure team)20:12:45

if you don’t know, then why say anything at all? :)

lwhorton20:12:15

well, i guess to be more clear (arghh the english language): i like the idea of nil meaning “look person, i dont even know what you’re talking about…“, which currently cant be represented in postgres (unlike datomic where you can simply omit something). and maybe the best way to do that is simply nil the field

eraserhd20:12:34

You could always make one Postgres table called "facts" with "e" "a" and "v" columns .... (jk, don't do this)

lwhorton20:12:23

between triggers, log tables, and a reified transactions table with some cleverness we’re getting close to about 25% of the power of datomic

ro620:12:24

Somewhere I read that the Drupal CMS basically does that, never confirmed it though.

shaun-mahood20:12:08

@lwhorton: Anecdotally, I've worked with systems that blow up hard if there's a null in the wrong place - particularly where it is either a vendor database and so I can't change anything, or if there are legacy or external systems that I'm integrating one of my databases with. I'm seriously considering stripping out nulls as part of my JDBC queries as a step towards migrating to Datomic and to make the Clojure code a little more consistent (though this may be a terrible idea).

lwhorton20:12:19

thanks for the insight @shaun-mahood. i’m working in an elixir system and also very much considering modifying the db connector to auto-strip nulls too. as alex said, if we dont know it why bother even declaring we dont know it.