Fork me on GitHub
#clojure
<
2017-07-22
>
cfleming03:07:58

@puredanger Are your deps slides available anywhere, until the talk is online?

alexmiller05:07:53

I'll put some stuff out early next week - slides, code, and some more info

alexmiller06:07:51

want to have a little more context than just dumping the slides. haven’t had a chance to do all that stuff due to the conf and today’s travel home.

cfleming10:07:48

Sure, no problem - thanks!

mjo324_5605:07:30

hi all, good morning

mjo324_5605:07:47

how do I check wether a process is alive?

mjo324_5605:07:57

i mean i launched a subprocess

mjo324_5605:07:03

whether with sh or conch

mjo324_5605:07:19

and i'd like to know whether it's finished or not

akiroz06:07:11

@mjo324_56 sh is a blocking call:

(sh "ls")
(prn "finished")

mjo324_5606:07:35

i did it like:

mjo324_5606:07:51

(defmacro try-noooo "Returns the result of evaluating e, or :noooo if it throws an exception." [e] `(try ~e (catch java.lang.Exception _# :noooo)))(defn is-proc-running[p] (if (= (try-noooo (.exitValue p ) ) :noooo) true false))

mjo324_5606:07:39

sorry about the formatting, havent figured this out yet on slack

mjo324_5606:07:32

the .exit-Value call throws an exception if the process is still running

mjo324_5606:07:39

that's the trick

akiroz07:07:51

@mjo324_56 If you use the built-in future to handle async, you can do this:

(def process-future
  (future (sh "./my-script.sh")))

(when-not (future-done? process-future)
  (prn "still running...."))

mjo324_5608:07:40

@akiroz that would do the trick, but i also need a timeout

akiroz08:07:12

@mjo324_56

(let [result (deref (future (sh "./my-script.sh")) 100 :timeout)]
  (if (= result :timeout)
    (prn "time limit exceeded.")
    (prn "process exited with: " result)))

mjo324_5610:07:35

you're the master!

mjo324_5610:07:39

that's far more elegant than my solution

hmaurer10:07:36

Hi! What is the preferred way to do input payload validation for a json api in Clojure? If possible a solution that offers both input validation and machine-readable errors to be consumed by the client. Hopefully I won’t start a 40min heated debate like yesterday 😄

misha10:07:50

I'd read one, though opieop

weavejester11:07:22

@hmaurer There are a few solutions. Compojure-API can use Schema or Spec. My own Ataraxy has spec validation in the latest version. There are also various validation and coercion libraries, but I think the most common approaches are Schema or Spec.

weavejester11:07:00

Spec isn’t designed around coercion so much, so I’m not sure how well it could handle that.

ikitommi12:07:47

@hmaurer wrote a post about the coercion & error messages some time ago: http://www.metosin.fi/blog/clojure-spec-with-ring-and-swagger/.

yonatanel13:07:01

I'm using integrant-repl and reading config from resources folder in my preparer function (`(read-config (io/resource "config.edn"))`) and yet it won't pick up new values for mongodb host when I change them, forcing me to stop and start the REPL. Does this ring a bell to anyone? Any common gotchas?

weavejester15:07:27

I'd need to see more of your setup. If you're performing a reset, your prep function will be hit

hmaurer13:07:17

Quick question: how can I pass a static function around? E.g. if I try to pass System/getenv around I get Unable to find static field: getenv in class java.lang.System

hmaurer13:07:28

(but (System/getenv "foobar") works)

hmaurer13:07:48

I could pass #(System/getenv %) around, but I assume there is a simpler way

noisesmith13:07:02

that's a static method, methods unlike functions are not first class objects supported by the vm

noisesmith13:07:20

you can't pass something as an argument that can't be put on the stack

noisesmith13:07:49

that's why you need to create a function, which is an object (with a static method that gets called if you apply it)

noisesmith13:07:02

so tl;dr there isn't a simpler way, that's the way you do it

noisesmith13:07:01

@leonoel that doesn't work with static methods

noisesmith13:07:25

and it's not any more succinct or efficient than creating a lambda in most cases

noisesmith13:07:42

(that's what it actually does, it creates a function)

leonoel13:07:52

oh right, my bad

hmaurer13:07:54

@noisesmith ah I see; I thought there might be something like that going on

rorysmith14:07:30

hi all, i am trying to understand how reduce works in the following example:

user=> (reduce str (repeat 3 "str"))
"strstrstr"
i don't understand why this returns what it does. can someone break down what reduce is doing here?

dpsutton14:07:26

@rorysmith head to #beginners and I'll walk you through it

hmaurer15:07:12

@weavejester yet another question on Integrant… Why is ig/suspend! necessary? Couldn’t you do all the logic in ig/resume? (naming it ig/swap or similar). It seems to only be used when hot-swapping for dev

hmaurer15:07:23

There are two steps: suspending the system then resuming it with a different config

hmaurer15:07:39

To do essentially one thing: hot-swap some parts of the running system

hmaurer15:07:51

(at least that’s the only use of suspend!/resume I have seen yet)

weavejester15:07:16

Consider the case of a web server. Suspend can tell the server to buffer connections until resume is called. After suspend we reload namespaces which can take several seconds.

weavejester15:07:35

Suspend and resume are only used for dev

hmaurer15:07:07

ah right, so you wouldn’t be able to hot-swap in one go because reloading namespaces while requests are being processed would lead to issues I assume

hmaurer15:07:41

Ring will buffer requests by default if the handler is a promise?

hmaurer15:07:02

well, the ring jetty adapter

weavejester15:07:53

Yes, because a promise blocks

hmaurer15:07:36

Ah, so if the passed handler is a promise it will try to deref, which will block. ok.

weavejester15:07:45

Suspend and resume are useful for keeping connections open and avoiding restart time during development.

weavejester15:07:01

Web sockets for example

beders16:07:46

How are you guys printing the guts of Java objects on the REPL? I'd love to be able to convert any java object to a graph of maps I can easily inspect. (similar to what the debugger in IntelliJ allows you to do)

beders16:07:30

wouldn't work on non-beans

beders16:07:02

I tried to make sense of clojure.reflect but before I spent my time on that - shall we say - sparsely documented lib, I was wondering if someone had a neat java-to-map fn

bfabry16:07:05

I think clojure.reflect/reflect basically does that iirc

noisesmith16:07:11

if I recall, bean used to be more general and didn't look for JavaBean specific stuff... let me see if I can find it

noisesmith16:07:53

+user=> (bean (java.util.Date.))
{:day 6, :date 22, :time 1500741217657, :month 6, :seconds 37, :year 117, :class java.util.Date, :timezoneOffset 420, :hours 9, :minutes 33}

noisesmith16:07:04

maybe it's that Date is a bean and I never realized it was?

bfabry16:07:12

yeah bean looks quite good

(bean (HTableDescriptor. (TableName/valueOf "foo")))
=>
{:familiesKeys #{},
 :columnFamilies #object["[Lorg.apache.hadoop.hbase.HColumnDescriptor;"
                         0x124eda1b
                         "[Lorg.apache.hadoop.hbase.HColumnDescriptor;@124eda1b"],
 :regionReplication 1,
 :memStoreFlushSize -1,
 :name #object["[B" 0x19dca84c "[[email protected]"],
 :tableName #object[org.apache.hadoop.hbase.TableName 0x6328ce6a "foo"],
 :metaRegion false,
 :rootRegion false,
 :compactionEnabled true,
 :maxFileSi... snip

noisesmith16:07:38

and you could start with bean's source code if you want something that isn't so JavaBean centric

cddr18:07:17

I don't understand this line in the docs about clojure.spec "Thus a bare (s/keys) is valid and will check all attributes of a map without checking which keys are required or optional." What is it checking if there are no specs to check?

(s/def ::foo (s/keys))
(s/valid? ::foo {:yolo 42})
=>true

cddr18:07:44

Ah I hadn't read enough. Namespace qualified keys will be checked against their specs

arohner19:07:47

Are there any good debuggers that work at the bytecode level? I have cider, but I have an infinite loop somewhere, so I need ‘pause execution’

noisesmith19:07:48

cider and cursive have this - to some degree, you need to turn off some clojure features for it to work well

noisesmith19:07:31

usually I opt for adding (swap! debug-atom conj {:context ::foo :data foo}) in the middle of some function and then use the resulting atom in the repl to figure out what's going on (which does require iterating sometimes to figure out which data you should even be tracking of course)

hmaurer19:07:30

@noisesmith is there an equivalent of Ruby’s “pry” in clojure? e.g. a way to open a repl at any point in a program, with access to local bindings?

arohner19:07:30

I thought cider’s debugger worked by instrumenting and evaling source?

noisesmith19:07:07

@arohner I didn't check recently but at one point they were actually using jdx bindings to get into the byte code level - when I saw people claim "cider can debug like cursive now" I assumed that meant that feature was working

noisesmith19:07:18

cursive definitely does byte code level debugging

noisesmith19:07:38

@hmaurer that's hard with a compiled language

noisesmith19:07:49

I don't think that exists, definitely not in a general way

arohner19:07:52

@noisesmith that doesn’t look to be in the latest release: https://cider.readthedocs.io/en/latest/debugging/

noisesmith19:07:41

@arohner oh, clearly I was misinformed and/or misunderstood what I was being told

noisesmith19:07:47

thanks for the clarification

arohner19:07:02

@hmaurer cider can do that by recompiling functions. Macros get access to all local variables in scope when compiling

arohner19:07:55

i.e. (let [x 1] (my-dbg (inc x)), if my-dbg is a defmacro, there’s an extra &env variable that contains {x 1}

hmaurer19:07:18

Well, I guess I should start using emacs then

noisesmith19:07:46

I find it easier to just use a macro that puts the locals in a hash map, then access it https://gist.github.com/noisesmith/3490f2d3ed98e294e033b002bc2de178

arohner19:07:49

a bytecode debugger can see the locals in scope, but IIRC the compiler throws away the variable names in the bytecode, so you just get foo_1, etc.

bronsa19:07:53

it doesn't

bronsa19:07:03

the compiler, I mean

bronsa19:07:11

the local names are right there in the bytecode

arohner19:07:53

and there’s locals clearing

noisesmith19:07:11

@arohner right that's the clojure feature I was t alking about - you can disable it

michaellindon19:07:18

Has anyone used visualvm to profile clojure code?

michaellindon19:07:23

I cant get it to work

noisesmith19:07:28

yes - it works but yourkit is much better

michaellindon19:07:30

is there any special setup in the project.clj to use YourKit? It seems a lot of feature are missing when i connect to my running repl

noisesmith19:07:17

you might need to add jvm args to allow the kind of connection it wants

noisesmith19:07:27

those would be the same args a java program would need

noisesmith19:07:56

I've had a lot of luck with using the yourkit agent

michaellindon20:07:40

seems to be working much better than visualVM

michaellindon20:07:01

its hard to make sense of though

michaellindon20:07:23

in the sense that it is hard to find which of my functions is taking the longest time amid all the clojure / java functions

noisesmith20:07:38

yeah - it takes a while to figure out, I have a long term plan to do a talk on this topic

michaellindon20:07:43

my code takes 7 days to run ^^

arohner19:07:29

try the sampler rather than the profiler

arohner19:07:41

profiler rarely works well for me

noisesmith19:07:06

but both are somewhat difficult because so much of profiling assumes that the class you are looking at is important, and most of the classes are going to be eg. clojure.lang.PeristentVector

arohner19:07:29

sure, but at least the sampler comes back with a result 🙂

noisesmith19:07:37

it's definitely doable though - I've had a lot of help from profiling

arohner19:07:38

profiler usually takes forever to instrument and then doesn’t work

michaellindon19:07:43

well, I've been following the guide written here: https://torsten.io/stdout/how-to-profile-clojure-code/

noisesmith19:07:51

@arohner oh, I haven't had that issue

noisesmith19:07:03

it needs more time to start up for sure though

michaellindon19:07:19

but it says Failed to Create JMX connection to target application

noisesmith19:07:35

then you need to add the args to java to allow jmx connect

hmaurer19:07:15

is there any good way to debug clojure without using a particular editor? (aka no cursive and no cider)

noisesmith19:07:43

that's where swapping data into an atom excels

noisesmith19:07:59

or you could use jdb - if you are familiar enough with clojure internals it can work

noisesmith19:07:01

but it's confusing

hmaurer19:07:23

@noisesmith funnily enough swapping data into an atom is the solution I came up with this morning

noisesmith19:07:33

there's an old tool for setting up a "debug repl" that kind of worked but it was funky

arohner19:07:00

@michaellindon are you specifically trying to get remote profiling working? Local clojure process should just work

noisesmith19:07:17

@hmaurer as a level up from swapping into an atom, I also dump the relevant part so I can use it in a unit test when I figure out what is going on - I made a small lib for this https://github.com/noisesmith/poirot

hmaurer19:07:49

@noisesmith oh yes, you sent that one to me already but hadn’t yet looked into it

noisesmith19:07:06

@michaellindon oh - weird - I haven't experienced that issue, not sure what that's about

michaellindon19:07:18

😞 Ive had this working on a remote process before... something about running it locally is not working

arohner19:07:46

ok, got jdb working acceptably well

noisesmith20:07:26

@arohner I just realized this would be relevant - if you send a signal you can make the jvm print all its stack traces (or you can use jstack from another terminal to get all stack traces by pid) and if you have a loop that isn't exiting a few stack traces should show where that's happening pretty quickly

arohner20:07:50

yeah, I did basically the same thing using jdb

noisesmith20:07:54

I forget the signal name but on *nix terminals its bound to C-\

arohner20:07:02

using threads and where

hmaurer20:07:16

Can someone please remind me of the name of the clojure function which does the following: (f g :a :b :c) => [(g :a) (g :b) (g :c)] ?

hmaurer20:07:44

ah I am stupid, I could just map

arohner20:07:45

looks almost like juxt

hmaurer20:07:55

but juxt is what I was looking for

hmaurer20:07:15

my example was just wrong

mobileink21:07:49

haha! didn't i tell you? :)

hmaurer21:07:38

@mobileink you did! that’s how I vaguely remembered there was a function for this, but I have the brain of a goldfish today so I couldn’t think of the name

dominicm22:07:12

Is there a form of resolved keywords which looks into refer'd vars?

(ns foo
  (:require [clojure.string :refer [blank?]]))

(println ::blank?)
;; (desired) => clojure.string/blank?
;; (actual)  => foo/blank?

hmaurer11:07:03

oh thanks, that looks interesting

bronsa22:07:08

keywords have no relationship with vars @dominicm

yonatanel23:07:30

@dominicm is it something you need for spec?

dominicm23:07:37

I figured it would be useful for that, yes

yonatanel09:07:50

How would you use it?

cfleming23:07:23

@arohner Cursive has a bytecode debugger, and you can definitely see the locals (as well as other normally invisible locals created by e.g. destructuring). It also does its best to help control locals clearing.

cfleming23:07:42

You’re right AFAIK that CIDER’s debugger is source transformation.