Fork me on GitHub
#beginners
<
2018-07-18
>
uwo00:07:09

Is there a way to gracefully close the source stream to (clojure.data.xml/parse input-stream) without causing

javax.xml.stream.XMLStreamException: ParseError at [row,col]:[3157,54]
Message: Stream Closed
        at com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl.next(XMLStreamReaderImpl.java:596)
        at clojure.data.xml$pull_seq$fn__16462.invoke(xml.clj:287)
(I’m calling .close manually in exception handling because I can’t wrap with with-open in this case)

uwo14:07:59

rookie mistake here: figured out what I was doing wrong. the step that was attempting to to spool data off the parse stream was attempting to call first after the stream had been closed

yqrashawn04:07:45

HI guys, a simple question. I use emacs with cider trying clojure. After I add a dependency in project.clj file. Should cider-restart download the dependency and re cider-jack-in?

tianshu04:07:14

@namy.19 you can add dependencies without restart the repl with clj-refactor. If you simply change the project.clj file, you should restart the repl. when the repl started, it will download the dependencies, no need to restart twice.

yqrashawn04:07:11

@doglooksgood but when i run cider-restart, there’s no new dependency in the ~/.m2 folder, and requiring new dependency in core.clj file won’t work.

tianshu04:07:21

I haven't use cider-restart. usually I just C-c C-q to close cider connection, and then C-c M-j to jack-in again.

yqrashawn04:07:01

Yes, that’s works fine. I see.

yqrashawn04:07:42

@doglooksgood Thanks! I’ll check clj-refactor.

yqrashawn04:07:30

Wow, thanks.

yqrashawn04:07:57

Just checked the cider-restart function, seems it only reconnect the repl, not killing repl and restart it.

yqrashawn04:07:27

I’ll definitely try clj-refactor

lepistane10:07:56

hi guys i am pulling some numbers from db using queries and i am constructing map like this

{:k1 (query1)
:k2 (query2)
... 
that is pretty inefficient in my mind and i had an idea instead of making 50 queries to get values for that map to create massive query which would return map or list of values that i will bind to keys later on is it possible to do something like that? this might be more sql question than clojure one

valerauko10:07:03

should be possible yes

lepistane10:07:14

any ideas how?

valerauko10:07:45

run the query and reduce the rows it returns into the shape you want

lepistane10:07:44

i am already something something like that i want do get all those values in 1 db call not in 50

delaguardo11:07:08

select (select ...) as k1, (select ...) as k2, ...

delaguardo11:07:39

strait forward solution but it is possible to optimize it later

lepistane11:07:57

this is what i needed

lepistane11:07:02

any hints about optimizations?

valerauko11:07:40

oh you have all-over-the-place queries?

delaguardo11:07:41

depends on a queries that you are running

valerauko11:07:20

i was thinking of something like

(reduce
  (fn [aggr row] (assoc aggr (:id row) (:value row)))
  {}
  (sql/select "select * from table limit 50"))

delaguardo11:07:49

if some of them selecting identical data but transforming them differently you can place one query into from block and apply different transformation functions in top select

👍 4
valerauko11:07:42

not sure if many transformed subqueries are better than many individual queries

delaguardo11:07:51

sometimes just placing subqueries in one query can give some ideas how to optimize it

delaguardo11:07:45

and in some cases subqueries can be optimized by DB engine itself )) (unfortunately, rare cases)

vinai12:07:14

I'm trying to get a grip on logging. I've got a ring based app. The log is spammed with useless entries, I would love to be able to filter noise. One way to do that is by the log level, but I actually would like to filter out some info level messages while leaving others intact. Just for context, if I do

(require '[clojure.tools.logging :as log])
(log/info "Foo")
it logs it straight out to my repl. On heroku I use papertrail to access Iog messages. Now, is there a way I can wrap the logger with a middleware like wrapper function?

jonahbenton16:07:25

From a beginner perspective, the JVM logging story is sordid and awful. To avoid most of that I would suggest timbre instead of clojure.tools.logging: https://github.com/ptaoussanis/timbre The difference is that timbre is a logging system, not a wrapper that delegates to a usually java-native logging system. It also is plain clojure (clojure/clojurescript), not something that consumes an underlying java library. So one can do middleware stuff and formatting output and so forth in a more clojurey manner. And it has explicit whitelist/blacklist configuration. Modern Java libraries of course write their own logs; they can usually be implicitly made to have their logging go to timbre by adding to project.clj [com.fzakaria/slf4j-timbre "0.3.2"] Timbre itself is not trivial, there are a number of knobs to understand, and it's not the fastest logger nor necessarily the most ops-friendly. But I've found it to be a useful point of control and simpler to reason about.

vinai17:07:12

Thank you very much for that!

sb17:07:49

Hi, I would like to check a good Clojure tutorial about CircleCi integration with tests, somebody know a similar repo on Github or a good blog post? (I read the CircleCI docs, I found) — mainly for Ubuntu (not Heroku)

delaguardo17:07:55

the best place to start - https://circleci.com/docs/2.0/language-clojure/ and after that - check out how circleci frontend is integrated with itself) https://github.com/circleci/frontend/blob/master/.circleci/config.yml (edited)

David Reno20:07:02

I’m trying to deploy the default compojure-api app to openshift. I created the app like this: lein new compojure-api myapi. I’ve only modified the handler statement to use port 8080: :ring {:handler myapi.handler/app :port 8080}. Questions: 1. where openshift examples call for deployment of a war file, should I be using something like this to produce the binary artifact: lein do clean, ring uberwar? 2. The produced file seems to be called target/server.jar, should I rename it with a war extension? 3. Am I going down the wrong road with binary deployments? Should I learn maven and approach this as a build-on-demand deployment?

jonahbenton20:07:03

This is a complicated question. It appears that RH docs frame deploying a JVM app to Openshift in JBoss terms. This is not technically necessary, but both Openshift and JBoss are RH products so there is a natural shared audience. For the purposes of this project are you using or do you need to use JBoss with Openshift, or are you able to use Openshift without JBoss?

David Reno21:07:24

I’m a noob at java app deployments so take this with a grain of salt: I don’t believe that I have any restrictions on deployment. I don’t need to use jboss/wildfly. I’ve successfully done this following these directions: https://github.com/openshift-evangelists/openshift-cookbook/issues/1 However, that uses the redhat specific redhat-openjdk18-openshift image which is not available (to my knowledge) or at least not deployed in our corporate openshift “origin” environment.

David Reno21:07:38

For the previous example, I used lein ring uberjar

jonahbenton21:07:45

yup, got it. am in transit so will respond more fully later but the short version is that an "uberjar" can be run with a command line of java -jar uberjar.jar, so if you can create a docker container image with that file and that start command, openshift will be able to run the image.

David Reno21:07:58

hmm, sounds like I need to get into creating my own container image. That’s where I’m wondering if I’m taking the wrong approach and should switch over to git/source deployment and let openshift build for me.

jonahbenton23:07:05

Yes, the goal is a container image, and it seems like using the oc build stuff with the OpenJDK templates is the right way to start. However, it looks like you will need to feed an uberjar to oc build. The builders it seems to have don't know how to take clj source from git and turn that into a jar artifact. So using

lein ring uberjar
should produce that .jar file for you, which you can then do
oc startbuild appname --from-file file.jar
to produce a deployable image. That said, it should be noted that producing the jar locally doesn't give you any provenance in the container image- unless you are exceptionally diligent one wouldn't be able to trace what's in the image to a particular commit in the clj repo.

David Reno16:07:35

Does lein do clean, ring uberwar in fact produce a “war”? The produced file is named target/server.jar. I’m following these directions and trying to use the file as a war file but getting errors on the wildfly console.

jonahbenton16:07:24

Those steps should work. It has to produce a jar of the application first, then include that jar in the war. The war I don't believe is written to the root of target, there should be something else there, maybe an uberjar directory?

David Reno17:07:02

➜  myapi lein do clean, ring uberwar
Created /Users/dreno200/Documents/dev/projects/myapi/target/server.jar
➜  myapi ls -R |grep -i war
➜  myapi

David Reno17:07:06

Not seeing the war file anywhere.

David Reno17:07:10

running this, I see differences on server.jar depending on having used uberjar/uberwar: jar tvf target/server.jar

David Reno17:07:00

It seems that server.jar is the default output regardless of creating a jar or war:

➜  myapi lein do clean, ring uberwar mywarfile.war
Created /Users/dreno200/Documents/dev/projects/myapi/target/mywarfile.war
➜  myapi

jonahbenton17:07:29

Is the name server.jar used anywhere in project.clj? The default naming should include a name and version, so I suspect the name is being overridden in project configuration.

David Reno17:07:59

default project file plus :port 8080:

(defproject myapi "0.1.0-SNAPSHOT"
   :description "FIXME: write description"
   :dependencies [[org.clojure/clojure "1.8.0"]
                  [metosin/compojure-api "1.1.11"]]
   :ring {:handler myapi.handler/app :port 8080}
   :uberjar-name "server.jar"
   :profiles {:dev {:dependencies [[javax.servlet/javax.servlet-api "3.1.0"]]
                   :plugins [[lein-ring "0.12.0"]]}})

jonahbenton17:07:42

Ah, there tis- :uberjar-name. Try removing that key-val pair.

David Reno17:07:48

now it creates target/myapi-0.1.0-SNAPSHOT-standalone.war. I don’t think anything will change for me though, I copy this into my wildfly directory and see this on the console:

17:22:27,394 INFO  [org.jboss.as.repository] (DeploymentScanner-threads - 2) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/b9/86413477a47e000b053e6c301f39292460d4cd/content
17:22:27,401 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-5) WFLYSRV0027: Starting deployment of "myapi-0.1.0-SNAPSHOT-standalone.war" (runtime-name: "myapi-0.1.0-SNAPSHOT-standalone.war")
17:22:38,210 INFO  [org.wildfly.extension.undertow] (ServerService Thread Pool -- 77) WFLYUT0021: Registered web context: '/myapi-0.1.0-SNAPSHOT-standalone' for server 'default-server'
17:22:38,224 INFO  [org.jboss.as.server] (DeploymentScanner-threads - 2) WFLYSRV0010: Deployed "myapi-0.1.0-SNAPSHOT-standalone.war" (runtime-name : "myapi-0.1.0-SNAPSHOT-standalone.war")
Looks good but I still only see the “welcome to wildfly” message on

David Reno17:07:16

seems to be getting deployed, I just don’t get any effective web page from it.

David Reno17:07:16

thanks for your help on this! I’m a total noob with jar/war deployment. Just trying to ensure that I can actually deploy this somehow before I dive too much into it.

jonahbenton17:07:22

Sure thing- so the log line "...Registered web context: /myapi-0.1.0-SNAPSHOT-standalone" suggests that wildfly will only route requests to that prefix to the deployed app. Try that path from a browser. That's a pretty common pattern for web app containers like wildfly (where the term "container" is completely unrelated to the docker/kubernetes/openshift use of the term)- the container is the host for a number of web applications, and has to be told how to route requests to each. It will use the application/artifact name by default, and configuration can be provided to give it an alternative.

David Reno17:07:39

ahh, ok, now I get something but it’s not good. Lots of errors starting with this:

17:39:24,373 ERROR [io.undertow.request] (default task-1) UT005023: Exception handling request to //myapi-0.1.0-SNAPSHOT-standalone: java.lang.NullPointerException: Response map is nil
	at ring.util.servlet$update_servlet_response.invokeStatic(servlet.clj:99)
	at ring.util.servlet$update_servlet_response.invoke(servlet.clj:90)
	at ring.util.servlet$update_servlet_response.invokeStatic(servlet.clj:94)
	at ring.util.servlet$update_servlet_response.invoke(servlet.clj:90)
	at ring.util.servlet$make_blocking_service_method$fn__12852.invoke(servlet.clj:113)
	at myapi.servlet$_service.invokeStatic(servlet.clj:1)
	at myapi.servlet$_service.invoke(servlet.clj:1)
	at myapi.servlet.service(Unknown Source)

David Reno17:07:51

and many more lines

jonahbenton17:07:11

So that means- the function "app" in the namespace "myapi.handler" is returning nil, rather than returning a map that has :status :headers and :body keys

David Reno17:07:23

btw, I renamed the war to ROOT.war (which I’ve seen somewhere else) and now it responds without a path, still errors

David Reno17:07:56

going DM so I can paste a snippet…

Caio Corrêa21:07:12

Hi there! What would be the best way to write integration tests to a rest api using ring? Do you guys have any examples or resources to suggest?

kennytilton22:07:07

@caicorre Not sure exactly what you mean by “integration tests”. Are you already using https://github.com/ring-clojure/ring-mock and looking for something more?

Caio Corrêa22:07:40

That’s exactly it. Thanks!!!

Chase23:07:14

i'm just curious about something. i think this REPL driven development is going to super cool and fun. but as i'm going along in basic tutorials and try things out like "eval-last-sexp" and stuff a lot of times it just says something like "nil." As I learn more about programming is that like a key part of developing programs? I'm not sure what it is actually telling me about the function. i assume i'm just not far enough along but I was just curious and wanted to chat.

dpsutton23:07:41

what is returning nil? i'm not sure i'm following what you are seeing

Chase23:07:21

an early example was something like this: (defn train [] (println "choo choo!")) and so i just say evaluate expression in my good ole spacemacs. It just says => nil. Now this is a comically trivially function but i figured the repl driven evaluation would be telling me what the function actually does. when i do send it to the repl it does show me it prints out "choo choo" and also nil.

dpsutton23:07:02

it's using the functions that the repl experience shines

Chase23:07:43

oh. like if another function calls out to my train function?

Chase23:07:45

that definitely makes sense to me.