Fork me on GitHub
#clojure
<
2017-02-20
>
seantempesta01:02:04

General workflow question for you guys. Is anyone developing locally, but running/testing remotely on a server? I’ve maxed out the available RAM on my laptop and can no longer run a Client + Server + Datomic locally. I’m using Intellij + Cursive for development. If you’ve found an ideal workflow please share! Right now I’m setting up an Ubuntu server and am planning on rsyncing on save + using a remote repl.

qqq02:02:12

@seantempesta : not sure how helpful this is: with emacs + dirac (instead of cider), it should be easy to run stuff remotely

seantempesta02:02:14

@qqq: I did emacs back in the day (and liked it), but once I started using IDE’s I can’t go back. Thanks though.

qqq02:02:59

@seantempesta: unless the ide has builtin remote dev support, rsync is probably the best approach

qqq02:02:17

actually, sshfs / fuse may work where you directly mount the remote fs as a local directory

seantempesta02:02:15

yeah, I’m looking into that now. Rsync seems a little inefficient in that it won’t know which files have changed so it’ll have to rescan on every save.

qqq02:02:02

wouldn't your src/ be < 10MB ?

qqq02:02:10

I can't imagine rsync file diffing being the bottleneck

seantempesta02:02:14

Welcome @davidtpriest! What brought you to clojure?

qqq02:02:15

unless you're programming on a raspberry pi

seantempesta02:02:22

Yeah…that’s a good point. I was thinking about my .git directory, but then again there’s no need to replicate it.

qqq02:02:21

this is unnecessary, but as an overuse of tech, you could probably use inotify or boot/watch to monitor when files change, and then remote copy them 🙂

tbaldridge02:02:48

@seantempesta a app I develop needs late amounts of memory. Doing remote nrepl with cursive is actually really nice

tbaldridge02:02:56

I'm not sure how it works but there's some amount of file syncing being done with Cursive. So you edit, tell cursive to reload a file in a repl and it sends your local copy over

tbaldridge02:02:25

That plus git works pretty well.

cfleming02:02:54

@tbaldridge @seantempesta Cursive does that based on changes in your local FS

cfleming02:02:33

So when you send a file, it’ll work out the dependent files which have changed since the last time you sent them, and send those before sending the file you actually told it to.

seantempesta03:02:23

@tbaldridge @cfleming: Oh? Rad! That’ll make things easier.

sophiago03:02:32

does anyone here have experience with the reducers library? i'm have a function where the bulk of the work is done by map where the number of function calls grow exponentially and the output doesn't need to be ordered so am benchmarking it with pmap r/map and a hand-rolled core.async version of map, but r/map is just returning some object code when i swap it out. do i need to use it differently somehow?

tbaldridge03:02:43

@seantempesta so you don't have to look it up, on the remote machine it's as easy as lein repl :headless :host 0.0.0.0 :port 4242

sophiago03:02:30

oh, answering my own question...it seems i need to wrap it in into, r/reduce, or r/fold. but in this case, i'm actually running it for side effects...i wonder if doall would work?

sophiago03:02:08

ah, nm. (into [] ...) works just fine. sorry for the waste 😛

sophiago03:02:23

oy, it seems to run as slow as pmap tho 😕

sophiago03:02:30

seems to be another case where parallelism is magnitudes slower...i think i underestimated how much i could tune the sequential version. it's at ~1µs now with pretty serious input so no way can i beat that with multithreading

tbaldridge03:02:33

@sophiago how long does each task in the map take?

tbaldridge03:02:58

I probably wouldn't try to make it parallel unless I can do it in batches of 10ms or so. 100-1000ms would be best.

sophiago03:02:36

the tasks themselves are very fast. there's just many of them. presumably if each were much more intensive it would be an ideal case for r/map

sophiago03:02:53

they also increase exponentially with the input. one of the parameters determines the number of calls to map. so i figured one of the upsides of reducers is i believe they fall back to the sequential version when appropriate?

sophiago04:02:15

anyway, it's just so fast right now running totally sequentially. i'm actually quite floored. i originally thought i made a mistake in benchmarking it when it was 100x this sequentially. and i still haven't tried some further optimizations

sophiago04:02:54

those being using forv and/or the vectorz library

sophiago04:02:52

one thing i found odd related to optimization is that it's not running any faster if i use (vector-of :short ...). nearly identical time

sophiago04:02:16

and i made care to not box them anywhere so unless i'm missing that i can't explain why that wouldn't be faster

placeboza08:02:27

Is there a simple solution to get my lein run output to support ansi colors in the text, in a windows console?

placeboza08:02:37

I think this is actually a java issue though

placeboza08:02:35

Google says I need a third party java library for it 😕

placeboza08:02:50

not worth it for this little test app

placeboza08:02:08

Ja, nevermind. Clearly a windows/java issue.

kauko10:02:11

I'm suddenly getting weird errors trying to start my repl. java.lang.NoClassDefFoundError that seems to point to clj-time internals. I've ran lein do clean, compile, repl multiple times, used git clean multiple times, and even deleted my .m2 folder. When I don't require clj-time into the failing namespace, the error occurs elsewhere. Anyone have any tips for me?

kauko10:02:25

I even tried cloning the project into a new repo and starting from there, same thing.

placeboza11:02:23

When you don't require clj_time , is it the same error that occurs elsewhere? (about clj_time.core.DateTimeProtocol) .. if not, what's the error then?

placeboza11:02:40

Might help id it

kauko11:02:40

You mean require it in the failing namespace?

kauko11:02:10

I tried that, and the compilation fails while compiling another namespace

placeboza11:02:20

with the same error though? about clj_time?

kauko11:02:57

Yeah, iirc. I'll try again in a sec. I killed all java processes and am trying to start the repl now

kauko11:02:13

if that doesn't work, I'll try a reboot. Which probably should've been one of the first things I try 😄

kauko11:02:19

The reboot didn't help. I'll try to remove the require to clj-time now

kauko11:02:07

Earlier I checked the md5 hash of the clj-time version that I have and compared it with a team mate, no difference there

kauko11:02:19

so it shouldn't be corrupted or anything

kauko11:02:42

also they can run the project just fine. "Works on their machines!"

kauko11:02:36

looks the same to me

dominicm11:02:43

@kauko the other standard thing to try: mv ~/.lein ~/lein_bak

dominicm11:02:52

make sure nothing in your local config is messing with it

kauko11:02:19

Good tip. Shouldn't be anything there, but I'll try anything at this point.

dominicm11:02:12

Sometimes plugins in your ~/.lein can bring in their own deps.

kauko12:02:37

Yeah that was it

kauko12:02:51

thank you! ❤️

dominicm12:02:24

"it's happening for me" = lein clean && mv ~/.lein ~/lein in my experience.

dominicm12:02:31

Many people have been driven insane by it 😛

dominicm12:02:42

actually, git clean > lein clean usually

dpsutton13:02:55

in calc distance, you are shadowing max and mod functions, so i would change those

dpsutton13:02:50

y-modulo being based on the modulus of x-count seems counter intuitive, which is already a modulus itself

dpsutton13:02:06

my guess is that there are much better variable names for these two concepts

placeboza14:02:55

sorry, work got in the way

placeboza14:02:25

glad it got worked out - I don't think I'd have figured it out that easily. Still too noob at this.

mruzekw15:02:27

I’m working with mount/`defstate` and cprop to load environment variables. I'm rather new to this set up (via luminus). I’m missing something because I get the following error when trying to access an env var:

mount.core.DerefableState cannot be cast to clojure.lang.IFn
I found this, but I’m having a hard time making sense of it: https://github.com/yogthos/clojure-error-message-catalog/blob/master/lib/mount/derefablestate-cannot-be-cast-to-ifn.md

danielsz15:02:47

@mruzekw The place you're trying to access an env var needs to be wrapped in a function.

mruzekw15:02:49

Could you give an example? This is akin to my current code:

(def auth (AuthAPI. (env :domain))

mruzekw15:02:00

How do I set the var for the module?

danielsz15:02:31

@mruzekw Put this in the start, for example

danielsz15:02:38

Or write (defn auth [] (AuthAPI. (env :domain)) and call it where appropriate.

mruzekw15:02:25

@danielsz I think I got it working with the second option. Thanks!

danielgrosse15:02:46

Why is the result 1 and not 1.5?

poooogles15:02:47

@danielgrosse >;; (quot m n) is the value of m/n, rounded towards 0 to the nearest integer.

poooogles15:02:38

(/ n m) might be what you're after

poooogles15:02:15

if you want a float out the end though make sure you pass a float in or coerce to one.

alexanderjamesking15:02:08

so

(quot 3 2)
is equal to
(int (/ 3 2))

poooogles15:02:56

>rounded towards 0 to the nearest integer.

poooogles15:02:15

doesn't int just drop everything after the .?

dpsutton15:02:49

looks like it

alexanderjamesking15:02:59

I think int rounds down to the nearest integer - I’ll try it with a few more examples

dpsutton16:02:58

does anyone know what might be wrong with this jtds connection object for jdbc?

(def conn {:subprotocol "jtds:sqlserver"
           :classname "net.sourceforge.jtds.jdbc.Driver"
           :subname "//server/database;user=user;password=password"})

dpsutton16:02:31

SQLException No suitable driver found for jdbc:jtds:sqlserver://

dpsutton16:02:49

i'm a .NET guy so i'm not exactly sure how the jtds connection strings should be formatted

curlyfry16:02:28

@alexanderjamesking int casts the number to an integer (a simple Java (Integer) x) behind the scene so like, @poooogles says everything after the . is dropped (the number is truncated)

akiel17:02:58

Hi. I like to build a macro called defhandler which instead of def’ing a var, registers the data/code in another way. Each handler has a name. Would you specify the name as symbol or keyword? I mean (defhandler foo …) or (defhandler :foo …)? I like to go the keyword route because it doesn’t def a var. It’s like in spec. Is this the right way?

quartz17:02:40

@akiel If you want to go the keyword way, you don’t necessarily need a macro but can probably (and should) do way with a function

quartz17:02:05

But that depends on the implementation details of said functionality, you can still have a macro for convenience’s sake 🙂

quartz17:02:02

You would need some kind of registry in which to save the handlers by keyword tho, maybe a simple hashmap could suffice

akiel17:02:22

@quartz The macro implementation currently builds a defmethod but customizes the code given.

akiel17:02:30

For the user defhandler looks a lot like defn except for the keyword instead of symbol.

quartz17:02:02

I see, I would personally go a more data-oriented way. Easier to reason about and mess with at runtime. Then I could write a macro with the semantics you’re looking for that would use the (hopefully) side-effect free functions under the hood

quartz17:02:39

But it really depends on what you already have implemented

quartz17:02:23

If you already have everything based on dynamic dispatch (defmethod) then it would require a bit of work to refactor, and maybe your current solution is fine for what you need 🙂

akiel17:02:59

I have not implemented much now. What I’m trying to do is to define handlers (business logic) together with middleware to apply and some other spec like data. Each handler can be presented as a map with one function in it. So it’s not pure data in the edn sense. What the macro currently does, it to generate a defmethod with the middleware already applied in code. I also could call the middleware at runtime from the data. You would go this route? Registering all the maps in an atom and running from there at runtime?

quartz17:02:47

You don’t need to go pure data if you don’t intend to send it somewhere else, using functions as objects is pretty fine

quartz17:02:16

But yes, I would try to aggregate all the info you need in a single hashmap probably and keep the functions as pure as possible

quartz17:02:33

Then you could maybe define some helper functions around an atom for convenience’s sake

akiel17:02:13

with functions you mean the map building and execution functions?

quartz17:02:41

Yes, your handlers/middleware would be functions. Pure if possible

quartz17:02:53

that take data in, modify it somehow and return it

akiel17:02:08

yes my handler and middleware functions are pure

quartz17:02:14

You can get some inspiration from existing projects like pedestal: http://pedestal.io/reference/interceptors

akiel17:02:04

yes my middleware is modeled after ring, except that I like to be able to define the usage of middleware right at the handlers. With ring, you usually have other places to put your middleware stack.

quartz17:02:39

That’s exactly what pedestal does with routes

quartz17:02:50

each route has a list of associated middleware/handlers

akiel17:02:56

Reason is that my handlers need special middleware each - so it’s not really the common middleware for many handlers

quartz17:02:23

So defining a route both with common and specific middleware would be as simple as

quartz17:02:24

common-middleware being a list/vector

akiel17:02:51

you can have two handlers at the end?

quartz17:02:13

Yes, because everything in pedestal is an interceptor and interceptors take a context map with a :request and :response keys associated with it

quartz17:02:00

an handler is just an interceptor that changes the response object

akiel17:02:13

I currently read the docs. It’s very nice that an interceptor can change the call stack!

quartz17:02:41

Technically this wouldn’t make much sense, because handlers in pedestal are functions that take the request as the first argument and are expected to return the response

quartz17:02:54

so the last handler would probably override the previous ones

tbaldridge17:02:04

@quartz @akiel interceptors are pretty much the closest thing you can get to a "reified stack" in Clojure.

quartz17:02:32

@tbaldridge that’s a very nice way to put it 😊

tbaldridge17:02:33

Where I define "reified stack" as: "taking the call stack and turning it into a value that you can manipulate and pass around like any other value"

akiel17:02:12

@tbaldridge Yes that is very powerful.

quartz17:02:36

We just love to use the verb reify don’t we 😄

tbaldridge17:02:57

and yeah, its super powerful, since you can pause execution (for async operations), modify the stack, and at all times debugging is much easier since you have a clear picture of what has been, and what will be executed.

akiel17:02:19

I thought pedestal interceptors would only add async on top of ring middleware, but that reified call stack thing is not bad.

tbaldridge17:02:51

That's why I don't like middleware-esque approaches anymore. I get really tired of getting exceptions inside a few functions that were comp'd together. Everything is opaque in that case.

quartz17:02:23

Indeed, have you checked Arachne @tbaldridge?

tbaldridge17:02:40

Yeah...I'm not a big fan of frameworks, or module systems though.

akiel17:02:14

Speaking of frameworks. Is the interceptor thing from pedestal available without HTTP?

tbaldridge17:02:43

@akiel yep, I know of some people who have used it with other things, like distributed queues, or other message-like things.

davidtpriest18:02:49

@seantempesta Thanks! Career change mostly. Wanted to get into web dev, started with python, was introduced to Clojure by a few developers. Was amazed. Went all in with Clojure and not looking back! 😀

akiel18:02:38

@quartz @tbaldridge Thanks for the pedestal pointer. I’ll try it in my use case.

quartz18:02:47

@akiel Good luck 🙂

quartz18:02:58

and have fun haha

grav20:02:01

How do I take-while and then one?

gfredericks20:02:28

either split-with and post-process that, or write it custom if you don't want to walk the sequence twice

gfredericks20:02:53

I bet a utility library or two has it

gfredericks20:02:35

I'd name it take-while+1

grav20:02:00

🙂 Well actually I don’t need anything but the one … so I guess drop-while together with (take 1) will suffice

dottedmag21:02:00

Is there a clearer way to express "if :selection key is in map, then it should be less than the length of :matching collection than

(fn [s]
  (or (not (contains? s :selection))
      (< (:selection s) (count (:matching s)))))
?

pesterhazy21:02:36

looks optimal

pesterhazy21:02:01

only thing you could do is use desturcutring

pesterhazy21:02:42

(fn [{:keys [selection matching]}] (or (not selection) (< selection (count matching))))

dottedmag21:02:36

thanks, much clearer.

uwo21:02:16

I’m running (tools.nrepl/start-server :port <port>)) on a remote server, and it appears to start however I’m unable to connect with lein repl :connect <host>:<port>. Moreover when I check to which ports are open with netstat -lptu my chosen port isn’t there. Any ideas what might be going on?

hiredman21:02:39

how does it appear to start? are you sure you are starting it on the right host? if you are starting it from a repl to a process running inside a docker container for example, the process will be listening in the docker container, not the host

camdez21:02:43

For the sake of sanity checking, I do see my nrepl server listed when I run that command:

tcp6       0      0 [::]:7888               [::]:*                  LISTEN      28446/java      

pesterhazy21:02:04

I'm guessing you need to bind to "0.0.0.0" to expose the port to connections from non-loopback interfaces, are you aware of that?

pesterhazy21:02:22

(plus it's probably best not to do that and instead use an ssh tunnel)

jstokes21:02:02

i think by default it will bind to 0.0.0.0

jstokes21:02:01

ah was looking at the clr one

pesterhazy21:02:35

it's localhost by default

uwo21:02:03

@pesterhazy I was not aware of that. it looks like it defaults to “localhost”. let me try

uwo21:02:27

ha. you already said that. wasn’t reading everythign

pesterhazy21:02:37

wall of text 🙂

uwo21:02:49

@pesterhazy darn. was hoping that would work, but it’s still not showing up on netstat

uwo21:02:49

@pesterhazy NVM! You are the winner!

uwo21:02:55

netstat doesn’t show me what I thought it was showing me. and now lein repl :connect ... works !