Fork me on GitHub
#clojure
<
2017-08-17
>
noisesmith00:08:39

any file under resources ends up on the classpath in the uberjar

noisesmith00:08:06

so if you make the right relative path for the file under resources, it should find it

rnagpal01:08:10

Are long let statements an antipattern or a code smell ?

noisesmith01:08:19

they are better than deeply nested evaluation in one form

noisesmith01:08:42

large functions in general are probably a sign that your design could be cleaned up

didibus05:08:00

Question: Is there a doseq-indexed? I want to know the iteration I'm on, I know there's map-indexed, but my use case requires side effects and I like list comprehensions.

joshjones05:08:11

@didibus (dotimes [n 100] ... ;; n bound to 0, 1, ...)

madstap05:08:01

You call it like (doseq [[i thing] (indexed things)] ,,,)

noonian07:08:25

Can also use map-indexed with vector to get pairs for doseq:

(doseq [[i ele] (map-indexed vector ["one" "two" "three"])]
  (println "element:" ele "index:" i))

kwladyka07:08:27

hey, as it sound like easy thing… how you use project.clj version in uberjar ? (io/resource "project.clj") show me project.clj from dependency instead of mine… (System/getProperty "app.version") doesn’t work with jar. Is it any simple way to achieve it?

misha08:08:09

;;; Jar Output
  ;; Name of the jar file produced. Will be placed inside :target-path.
  ;; Including %s will splice the project version into the filename.
  :jar-name "sample.jar"
  ;; As above, but for uberjar.
  :uberjar-name "sample-standalone.jar"

kwladyka08:08:38

sure, but i want use it in app for log purpose

misha08:08:12

oh, did not read question thoroughly opieop

kwladyka10:08:47

@misha thx, looks promising

blmstrm12:08:33

I meant so say contains? returns false.

roklenarcic13:08:15

are you sure that key doesn't include a space or something

blmstrm13:08:39

@roklenarcic Yeah, I’ve verified that it doesn’t contain any spaces. I read the data from a file with clojure.data.csv and use semantic-csv to turn the data into maps. I use semantic-csv in production daily without this problem. In my original file the fx_date column is the first. I tried switching places of the first two columns and now the :fx_date key works. It seems somehow the first column of my file isn’t read properly.

blmstrm13:08:06

@roklenarcic I’ve found the problem, or so I think. There was a BOM in the file I tried to parse,once I removed that it worked. Thanks for taking your time. 🙂

zignd14:08:52

hi there, i'm new here. @beoliver have you tried profiling the mongodb calls, logging the time taken on the calls? maybe they are the ones causing the inconsistent CPU time results

beoliver14:08:24

@zignd - not yet... was hoping i was missing a flag 😄

dadair15:08:52

Is there a built-in macro for creating {:a a :b b :c c} given input(s) [a b c]?

zylox15:08:46

i wouldn't expect that you need a macro for that.

eriktjacobsen15:08:51

@dadair (into {} (map (juxt keyword identity) ['a 'b 'c]))

dadair15:08:45

I thought I'd need a macro because I don't want to have the symbols a, b, c, as vals, I want to, say, return a map of let bindings with their names as keywords. Say: (let [a 1, b 2, c 3] {:a a :b b :c c})

eriktjacobsen15:08:09

might this help?

(defmacro local-context []
    (let [symbols (keys &env)]
        (zipmap (map (fn [sym] `(quote ~sym)) symbols) symbols)))
(from joy of clojure: https://coderwall.com/p/xayyvq/break-point-in-clojure)

andrea.crotti16:08:10

I started to use migratus to do database migrations and it's quite cool

andrea.crotti16:08:36

the only thing is that in initial phase (when there isn't really data around to keep) yet, I end up creating lots of very small migrations

andrea.crotti16:08:07

which is good in a way but maybe unnecessary and spreads around the schema a bit too much, making it less readable

andrea.crotti16:08:32

The good thing about some other more crazy ORMs is that they let you squash all the migrations

andrea.crotti16:08:13

which however doesn't really seem feasible with raw SQL migrations, how do you guys do that?

jeff.terrell16:08:30

@eriktjacobsen - That is amazing stuff, about adding a break point. :thinking_face:

cddr16:08:47

@andrea.crotti The way that rails does this is to export a representation of the schema and associates it with the "migration id" that was used to generate it. e.g. http://edgeguides.rubyonrails.org/active_record_migrations.html#types-of-schema-dumps In clojure, I haven't worked with a database that had enough changes for this to be a problem but that solution worked pretty well when I was working on a rails project. Not sure if any of the clojure migrations libraries support that feature but it would be a cool addition IMO.

cddr16:08:57

You check this representation into your repo and then when you run "migrate up" against a clean db, it just loads the db from exported representation rather than running through all the migrations.

cddr16:08:25

Actually looks like flyway supports this kind of thing https://flywaydb.org/documentation/command/baseline

tbaldridge16:08:42

As a person who used to spend the majority of their time writing stored procs and doing SQL optimization, I don't trust any migration software. In the end what worked for us is to keep the migrations as versioned scripts. "To go from V1 to V2 run this script". It has to be run by hand, but frankly if I run it by hand and it fails I know the state of my DB.

tbaldridge16:08:25

The worst thing is when you run an automatic migration, something fails, and you're left wondering "great...what's the state of my DB? And is the data completely corrupted now?"

tbaldridge16:08:27

MS SQL also had a really nice "export schema as SQL" function. So we'd do both, a "bootstrap.sql" file to run if you were starting from scratch. And then timestamped files that represented migration commands from the previous version.

andrea.crotti17:08:20

Mm I see, but with postgres for example you can wrap even all the schema changes in transations right?

andrea.crotti17:08:58

Not sure if migratus does that already but it spittle be easy to ensure you don't end up in funny states

pesterhazy17:08:13

I agree with @tbaldridge often the best migration framework is none at all

pesterhazy17:08:46

with pgsql i've used this:

pesterhazy17:08:48

create table if not exists migrations (
  key text CONSTRAINT pkey PRIMARY KEY
);

create or replace function idempotent(migration_name text,code text) returns void as $$
begin
if exists (select key from migrations where key=migration_name) then
  raise notice 'Migration already applied: %', migration_name;
else
  raise notice 'Running migration: %', migration_name;
  execute code;
  insert into migrations (key) VALUES (migration_name);
end if;
end;
$$ language plpgsql strict;

pesterhazy17:08:40

then you make sure a migration is invoked only once like this:

do $do$ begin perform idempotent('V33751695__products_fix', $$

ALTER TABLE products RENAME COLUMN cost TO cost_amount;

$$); end $do$;

pesterhazy17:08:04

that gives you a minimal migration framework - you can re-execute safely all .sql files in migrations/ when you migrate

dominicm17:08:00

Slightly contrasting view: it's nice not to have that snippet of sql that diverges. Even that in a library would be sufficient for me to not know about migrations from a sufficiently high level.

pesterhazy17:08:38

@martinklepsch I'm putting it on my list 🙂

ghadi17:08:53

Haha plpgsql has eval?

pesterhazy17:08:01

also I really like the custom quoting in the language: "bar" = $foo$ bar $foo$, or $baz$ bar $baz$ etc

pwrflx17:08:49

in Cider, how to navigate to a random namespace or function? Eg. I just want to look at the source of clojure.string, but I don't have it written like that any of the sources..

pwrflx17:08:38

hm. there is a namespace browser reachable via the dropdown menu, that's better than nothing, although fuzzy search would be better

ghadi17:08:35

@pwrflx should be C-c . if your cursor is upon the symbol

pwrflx17:08:34

C-c C-. you mean?

pwrflx17:08:27

@ghadi the cursor is not at a symbol.. what I'd like to do, if for example I remember, that that there was a function named blank? somewhere. I'd like to just type blank? and then it would show a fuzzy searched list, containing "clojure.string/blank?" as a result

ghadi17:08:46

sorry, no idea. there's a #cider room

pwrflx17:08:18

@ghadi thanks, I'll try there

pesterhazy17:08:44

@pwrflx M-x cider-apropos?

pwrflx17:08:06

@pesterhazy yes something like that, but that opens too many separate windows.. I'd prefer a fuzzy matched list.. similar like how class name works in Eclipse or IDEA

pwrflx17:08:57

seemingly there is a ticket to add a default action to cider-apropos, so at least it will jump to the source with one less intermediate window: https://github.com/clojure-emacs/cider/issues/1646

andrea.crotti18:08:28

mm well @pesterhazy all the migration tools normally have a migrations table which keep track of which migrations were run

andrea.crotti18:08:41

and if the migration itself and the change to that table are done in a transaction

andrea.crotti18:08:52

isn't that enough to ensure it only runs once?

andrea.crotti18:08:48

the other thing is that for example in this way I can write a test fixture very simply

(defn setup-db [f]
  (migratus/migrate config)
  (f)
  (migratus/reset config))

(t/use-fixtures :each setup-db)

pesterhazy18:08:49

The point of my snippet was that you can do something similar without a framework. I tried flywheel and didn't like it at all

andrea.crotti18:08:10

so I know that any function using the db will have everything migrated correctly

andrea.crotti18:08:58

well sure that's a good point, but a small framework like migratus it's still small enough to not be overwhelming and gives you a lot of nice tools

abnercarleto18:08:44

Hi guys, I need help about RESTful patterns I need to create a service wich send work days and return a date like: GET /due_date?work_days=2 and respond with: { due_date: '2017-08-21' } but this is not persisted what is the correct http verb and uri for this endpoint?

abnercarleto19:08:39

My question is about te correct RESTful pattern

zignd19:08:12

The pattern followed and probably the correct is to use only the resource name in the URL followed by some inner properties separating by slashes /, you should also use GET when you want to retrieve a resource, therefore, you're almost correct in your example. The only thing I would change is the query string section into something like so: GET /due_date/work_days/2. Most people use this pattern.

deg19:08:25

My machine gets extremely slow (freezes for many seconds) when I have a few REPLs running. I have 32GB RAM, so I assume I have enough memory. But, I notice that each java process is using about 12GB of VM. My guess is that the maximum VM size of each REPL is a function of total memory, and my large memory is simply causing each Java instance to grow very large, rather than GCing. Is this correct? and, can I improve performance by limiting the size of each instance?

ghadi19:08:18

yeah set -Xmx on each. You running docker?

deg19:08:04

Running Cider+Emacs on a CLJS program, so (at least) two instances there.

deg19:08:45

Plus, lots of other cruft running on the machine: a Windows VM that also grabs about 10GB of VM, and many Chrome windows.

deg19:08:20

What is a reasonable -Xmx to set? I assume I can specify it somewhere in project.clj?

deg20:08:28

@ghadi - Ok, I set :jvm-opts ["-Xmx2g"] at the top level of my defproject. It seems to have partially worked. One java has dropped to VM=6.7GB, but the other is still at VM=11.2GB, I assume that something (nrepl?) was not affected by the setting? And not sure why the affected one did not drop down to closer to 2GB.

adambard20:08:15

It seems there exist both JVM_OPTS and LEIN_JVM_OPTS environment variables, perhaps playing with those a bit might be informative (via https://github.com/technomancy/leiningen/blob/master/sample.project.clj#L510)

adambard20:08:54

I'm not an expert, but it seems likely to me that leiningen itself wouldn't use the :jvm-opts project setting, since it couldn't read it before it was started up.

baritonehands20:08:20

I've got a REPL middleware question: Say I wanted to add some middleware to add some restrictions to what you can eval. How difficult would that be?

baritonehands20:08:40

More context: I'd like to add an annotation and only allow operations on annotated classes

lemontea14:08:32

I think you would need to analysis the source code sent to nrepl somehow without blindly running it?

lemontea14:08:23

or you mean an annotation added to the nrepl request message itself? (which should be nearly trivial - just extract it)

baritonehands18:08:25

Continuing question from yesterday/earlier today, I want to analyze the clojure source in the REPL, and only execute if you're allowed to

baritonehands18:08:12

Like, check if the Java instance has an annotation, and only allow annotated classes to execute

deg20:08:20

@adambard . Thanks, but adding :lein-jvm-opts did not seem to make any further difference.

noisesmith20:08:53

it won’t work as a key in the project

adambard20:08:55

you'd need to do it in your environment before spinning lein up, if the other thing I said is in any way correct

noisesmith20:08:57

it has to be an env var

adambard20:08:11

e.g. LEIN_JVM_OPTS="-Xmx2g" lein repl

deg20:08:21

The good news is, at least for my immediate problem, just setting :jvm-opts was enough to make the machine more responsive.

noisesmith20:08:25

@deg lein itself runs only via the jvm, it can’t use an option for its jvm that it would need a jvm to parse

noisesmith20:08:43

which is why you then need the env var

noisesmith20:08:11

(for startup options at least)

deg20:08:39

But, I'd like to get that working too. I don't think I can do it easily on the command line, since I'm launching from inside emacs. But, I'll set it globally, I guess.

noisesmith20:08:03

it’s that simple

deg20:08:15

Oh (head slap)

ghadi20:08:22

@deg your VM=whatever measurement might not be accurate.

noisesmith20:08:28

(or maybe it’s set-environment-variable - shouldn’t take long to find the thing)

ghadi20:08:36

jps -lvm to ensure the command line flags to each jvm

ghadi20:08:23

there's also jstat -gccapacity $pid for lower level information about heaps

deg20:08:32

Cool, I did not know about jps. Or M-x setenv

adambard20:08:47

@ghadi this easily the most helpful answer, thanks for teaching me that that command exists 🙂

ghadi20:08:14

no problem!

deg20:08:12

Would I be showing my age if I said that the first Lisp I used professionally ran on a machine with a 16MB total address space? So, a small part of me is REALLY amused by this whole conversation!!

deg20:08:13

(For that matter, the first Lisp I used as a student was on a machine with 256KW address space, so 16MB was pretty expansive!!

dpsutton20:08:29

@deg this will let it prompt you if you like

(defun jack-in-with-memory (gb)
  (interactive "nGB of ram for lein")
  (let ((cider-lein-command (format "LEIN_JVM_OPTS=-Xmx%sg lein" gb)))
    (cider-jack-in)))

deg20:08:32

Thanks. (in my case, cider-jack-in-clojurescript, but same difference)

dpsutton20:08:00

i don't know enough about interactive arg programming but i'm sure there's a way to get it to use sensible default and prompt on prefix

deg20:08:05

@dpsutton Does that command work as is for you? I get an error "The LEIN_JVM_OPTS=-Xmx2G lein executable isn't on your 'exec-path'"

dpsutton20:08:28

ah dang, yeah, i forgot this is before it does the check that lein is present

dpsutton20:08:07

there's a command-global-opts but those options go after lein, not before

dpsutton20:08:11

if you want to submit a patch there's this line in cider-jack-in: (cmd (format "%s %s" command-resolved params))) and if you add another %s in there and make a var for environmental params you'd be good to go

dpsutton20:08:49

that's the actual command will be invoked. (and its the reason why we're failing command-resolved is not resolved with all the junk at the beginning)

deg20:08:18

Thanks! If I have some slack time, I'll try to do this. What project is it in? (But, realistically, I've got a heap of learning curve before I'm comfortable touching the Cider emacs code, so I probably won't be able to do this)

dpsutton20:08:43

jump right in! make it your own

deg20:08:46

🙂 I'm not saying that I won't... but I've done too many of these the past few days and gotta focus a bit on the main path. That said, if no one else jumps in, I'll try to get to it.

dpsutton20:08:47

dang, i actually might really want that as well. there are a bunch of env variables set in a project here at work

michaelblume21:08:37

https://clojure.org/guides/getting_started says that I can get a Clojure REPL just by running java -cp clojure-1.8.0.jar clojure.main

michaelblume21:08:01

but if I try this with a jar built from master I get a complaint about specs being missing

michaelblume21:08:26

is there a simple way around this?

noisesmith21:08:16

you need to add the spec libs to your classpath

noisesmith21:08:45

I’m forgetting where this is documented

michaelblume21:08:13

I tried java -cp target/clojure-1.9.0-master-SNAPSHOT.jar:~/.m2/repository/org/clojure/spec.alpha/0.1.123/spec.alpha-0.1.123.jar:~/.m2/repository/org/clojure/core.specs.alpha/0.1.10/core.specs.alpha-0.1.10.jar clojure.main before I gave up

bfabry21:08:54

@michaelblume this is a big part of the reason for the creation of toosls.deps.alpha, the clj script and the clj installer. because the next version of clojure has external dependencies

Alex Miller (Clojure team)21:08:17

@michaelblume that looks about right - did that not work?

michaelblume21:08:17

ah, I forgot the java executable doesn’t understand the meaning of ~

michaelblume21:08:21

expanding those out works

Alex Miller (Clojure team)21:08:30

was just gonna say that

michaelblume21:08:31

$ java -cp target/clojure-1.9.0-master-SNAPSHOT.jar:/Users/michael.blume/.m2/repository/org/clojure/spec.alpha/0.1.123/spec.alpha-0.1.123.jar:/Users/michael.blume/.m2/repository/org/clojure/core.specs.alpha/0.1.10/core.specs.alpha-0.1.10.jar clojure.main
Clojure 1.9.0-master-SNAPSHOT
user=>

Alex Miller (Clojure team)21:08:49

the idea here being that in 1.9, you will install clojure (in a system-specific way), then be able to just do clj to get a repl

Alex Miller (Clojure team)21:08:59

(plus have a bunch of tools to do other stuff)

deg23:08:03

@alexmiller That looks VERY cool!!! I assume there is already a plan in place for how it will play with CLJS too?

Alex Miller (Clojure team)23:08:39

Nope, this is just Clojure

Alex Miller (Clojure team)23:08:28

The problems are different in ClojureScript and there are things like lumo and planck

dafabebaca23:08:54

How can I add cors support in pedestal I've added

::server/allowed-origins {:creds true :allowed-origins (constantly true)}

dafabebaca23:08:04

But id doesn't seem to work

dafabebaca23:08:26

Neither is:

::server/allowed-origins [""]

dafabebaca23:08:43

The whole function is generated by lein new pedestal:

(defn run-dev
  "The entry-point for 'lein run-dev'"
  [& args]
  (println "\nCreating your [DEV] server...")
  (-> (lacinia/pedestal-service hello-schema {:graphiql true}) ;; start with production configuration
      (merge {:env :dev
              ;; do not block thread that starts web server
              ::server/join? false
              ;; Routes can be a function that resolve routes,
              ;;  we can use this to set the routes to be reloadable
              ::server/routes #(route/expand-routes (deref #'service/routes))
              ;; all origins are allowed in dev mode
              ;::server/allowed-origins [""]
              ::server/allowed-origins {:creds true :allowed-origins (constantly true)}
              })
      ;; Wire up interceptor chains

      server/default-interceptors
      server/dev-interceptors
      server/create-server
      server/start))

mfikes23:08:06

FWIW, @deg, if you create a deps.edn with

{:deps {org.clojure/clojurescript {:type :mvn, :version "1.9.908"}}}
Then clj -m cljs.repl.node starts a ClojureScript REPL.

mfikes23:08:36

You can also use clj -m cljs.repl.nashorn if you'd prefer to avoid Node.js entirely

deg23:08:24

@alexmiller @mfikes These are definitely great steps, so don't misunderstand my carping, but it is really difficult for new user to grok the setup needed for CLJS, so it would be great to direct some loving in that direction too.

seancorfield23:08:17

That seems to be down to @dnolen more than @alexmiller tho' @deg ?

seancorfield23:08:49

(And, yes, I agree -- my team tried to build some stuff with cljs a few years back and gave up due to the complexity/immaturity of tooling... and we haven't gone back yet)

dnolen23:08:18

No immediate plans, @mfikes point seems fine for now

seancorfield23:08:58

Tooling has improved a lot since then but it still seems a steep cliff -- based on feedback I see in the community from folks struggling to get cljs projects up and running.

dnolen23:08:39

We don't really get complaints much. Figwheel nREPL mostly works

dnolen23:08:58

And the basic stuff is way simpler than that

deg23:08:46

That's the easy part. It gets tricky -- very fast -- when you need to make any changes to a project.clj, or even when you want to figure out best template from which to start a project.

seancorfield23:08:14

Yeah, that's the typical complaint I see.

seancorfield23:08:51

(I don't care -- it'll be a year at least before I venture back to cljs... I might care then :) )

dnolen23:08:03

People complain about the same with Clojure

dnolen23:08:11

Nothing special here

deg23:08:20

Dave, you are doing incredible work but I think you may be too close to the project and too expert to really feel the pain points.

dnolen23:08:35

I listen to complaints all the time

dnolen23:08:42

And we respond to them

dnolen23:08:02

That's literally my job

dnolen23:08:52

Anyways if you have an idea propose something

dnolen23:08:59

And we'll think about it

deg23:08:03

Point taken. And appreciated. But, there are still some real issues. Some of them are hard; some just need a few lines of documentation.

dnolen23:08:20

Sure contribution welcome

dnolen23:08:32

Ideas and PRs

deg23:08:54

I'm trying to do my small bit. I put up a google sheet yesterday, trying to summarize the different templates for starting a CLJS project.

dnolen23:08:23

But post something somewhere not Slack

deg23:08:34

K. It's very late here too. But, would love to discuss privately, maybe early next week.

deg23:08:39

I'll reach out to you then?

baritonehands18:08:25

Continuing question from yesterday/earlier today, I want to analyze the clojure source in the REPL, and only execute if you're allowed to

baritonehands18:08:12

Like, check if the Java instance has an annotation, and only allow annotated classes to execute