Fork me on GitHub
#beginners
<
2018-10-19
>
Michael Fiano01:10:40

Is it possible to read at the granularity of bits with a Java InputStream? My reading suggests possibly not. In that case, is there anything else available if I wanted to read bits and bytes of arbitrary sizes from a file stream, which may even cross byte boundaries?

markmarkmark01:10:11

hm, though that might still only allow you to get at the byte level

markmarkmark01:10:54

you could read as many bytes as you want into a byte[] or a ByteBuffer and use that to make a BitSet

markmarkmark01:10:58

BitSets are pretty clunky but they allow you to get specific bits rather than being limited to bytes

Michael Fiano01:10:19

It also assumes data is in network byte order

markmarkmark01:10:04

you can change the byte order with a ByteBuffer

markmarkmark01:10:28

so presumably you could read it into a ByteBuffer, change the order to what you need and then put it in a bitset

Michael Fiano01:10:12

Alright, I'll read about that. Thanks. I havent touched Java in 20 years...this could get interesting

Michael Fiano02:10:31

If I depend on a Java library, under what namespace would it be accessed by?

Michael Fiano02:10:46

Not sure how to access it or add it to my ns

seancorfield02:10:27

(answered in DM)

CyberSapiens9705:10:06

Do you guys use Spec for creating automated tests for functions? Or create unit tests by yourself? Or both? I know this is a too general question, it might depends on the project maybe? But I’m wondering how good is to use Spec for automated tests

jsk07:10:36

Hi I have a question, I’m writing a program that takes samples of data coming into a web app. If I have some data objects trying to wrap that concept something like

(defn sample [source time quality] 
    {:source source
     :time time
     :quality (quality->metric quality)}
and I start passing that to many functions that extra data from it (:time sample) and I do that in a number of places… does one run into problems with that? Say if you want to change :time to :timestamp.

CyberSapiens9707:10:43

@jskulski I’m a beginner on CLJ too, but I think in that case, I would use a fully qualified keyword

CyberSapiens9707:10:58

And could maybe, create a function that only extracts :time. And use that function everywhere you need to extract it. So you don’t have to change code in a number of places

jsk07:10:39

I hadn’t heard about fully qualified keywords. I’ll look into them.

jsk07:10:41

Your next suggestion was the way i was leaning, but I wasn’t sure. I have done a lot of OO and primitive obsession can be tricky. So I tend to wrap stuff like that in classes/functions.

jsk07:10:57

Not sure if i just need to get more comfortable with raw data, which clj seems to like

CyberSapiens9707:10:35

Well, now I’m learning about Clojure Spec Library, it may be suitable for you, because you can specify any incoming data and use the specification for testing, data validation and other things

CyberSapiens9707:10:36

I tend to create functions for primitive/fundamental tasks, that I know I’ll use a lot, and then I use them to compose the tricky ones

CyberSapiens9707:10:41

Such as extracting some data into nested structures to perform some Boolean check, those things that you know you would be repeating to much, so it’s suitable for creating a function. Also even creating a high order function if you find yourself doing little different versions of a certain function...

andy.fingerhut07:10:13

Creating a function just to do (:timestamp foo) to avoid repeating :timestamp seems a little odd to me, but I have less experience maintaining significant Clojure code bases over long time periods. Text editors are pretty good at search and replace on :timestamp if you want to change it, if you don't use that as the prefix for multiple different keyword names.

andy.fingerhut07:10:32

And probably some refactoring tools like in Cider and Cursive might even handle that case well.

andy.fingerhut07:10:27

Namespace qualified keywords are a good way to avoid colliding with anyone else, as long as you pick a globally unique namespace name, e.g. based on a DNS domain name you or your org owns.

CyberSapiens9707:10:29

Oh, yeah, maybe for a tiny code such as (:keyword foo), refactoring with some tool seems more reasonable than creating a function :thinking_face:

CyberSapiens9707:10:59

I’ve seen people complaining about creating constants for keywords, because it might obscure what the value actually is inside a function, but I don’t see any problems doing that too, at least for me...

Wagk09:10:41

Hello, is it possible to declare a namespace that does not require the folder component?

Wagk09:10:18

I (believe) that src/module/main.cljs can declare something like (ns module.main ...)

Wagk09:10:34

is it possible to do something similar with src/main.cljs?

Wagk09:10:17

or I could be wrong on the first assumption too since it's not compiling for some reason

Em09:10:26

Good morning friends, does someone know how to cast a java class in Clojure. Basically, we want to do:

JavascriptExecutor js = (JavascriptExecutor) driver;
    js.executeScript("window.scrollTo({ bottom: 0,top : document.body.scrollHeight, behavior: 'smooth'})");
and the clojure code:
[driver]
  (.execute-script driver "window.scrollTo({ bottom: 0,top : document.body.scrollHeight, behavior: 'smooth'})"))

moxaj09:10:10

@emily there's no need to cast, but if you want to avoid reflection, you can typehint driver with ^JavascriptExecutor

moxaj09:10:17

like [^JavascriptExecutor driver] ...

Em09:10:17

Oo, ok. If it doesn’t need to be cast than my error is probably elsewhere. thank ya

Arkrait10:10:49

Is it ok to ask questions about exercism tasks here?

Arkrait10:10:12

I can't figure out why all the tests are passing except the one with the number 21897142587612075, for some reason the last digit in the result is not 5 but 6

jumar11:10:17

@UDK4U44BG I think you're suffering from imprecise representation of floating point numbers. Try this (see usage of the BigInteger's pow method: https://docs.oracle.com/javase/7/docs/api/java/math/BigInteger.html#pow(int))

(defn armstrong?
  ""
  [num]
  (let
      [numlist (digits num)
       squaresum (apply + (map #(.pow (biginteger %) (count numlist)) numlist))]
    (if (= (long squaresum) num)
      true
      false)))

grierson12:10:46

Just watched 'Simple made Easy' where Rich talks about Queues making systems much simpler and if you are not using them then you are missing out. So where should I start when using Queues with Clojure?

Alex Miller (Clojure team)12:10:30

core.async channels are one option

❤️ 4
Alex Miller (Clojure team)12:10:09

This idea is expanded on in his talk Language of the System by the way

grierson12:10:33

Ok, Thank you.

borkdude14:10:34

Just made a simple in-process pub/sub event system in couple of lines of code using core.async

noisesmith16:10:23

even with ByteBuffer you can only do byte boundary data, to do smaller packings you need bitwise operations

Michael Fiano19:10:29

Does anyone know of any libraries for character encoding? If I have a Java byte array, I'd like to decode it into a string of whatever character encoding was used. Currently I'm just doing (String. byte-array) for UTF-16.

jsk19:10:12

@cybersapiens97 @andy.fingerhut thanks for the thoughts! (i got tired and wandered off to bed ;))

andy.fingerhut19:10:55

@mfiano Java has java.lang.String method getBytes with several different variants you can check out the docs for.

andy.fingerhut19:10:14

Hmm I may have misled you there -- that method may go in the opposite direction you wanted.

Michael Fiano19:10:25

@andy.fingerhut it seems the String constructor accepts a second argument specifying one of the built-in charsets.

andy.fingerhut19:10:03

Yes, that is what I've found in some quick searches, too.

Michael Fiano19:10:22

Thanks for checking. I'm still getting used to navigating javadocs :)

dpsutton19:10:41

you mean the java hyper spec i'm sure 🙂

andy.fingerhut19:10:30

I was about to say that that reference goes over most people's heads, but then I guess Michael is one who knows it.

dpsutton19:10:02

i'm absolutely positive he is

Michael Fiano19:10:39

So if I'm reading byte arrays and converting them to integers, does it make sense to use an integer type 1 larger than the one specified, since the JVM only has signed integer types, and the number I am decoding may be unsigned and out of range for the same number type? Only thing I can think of is to use a number type 1 larger than the one specified, or use BigInts for everything

hiredman19:10:19

it depends what you are doing with the numbers

hiredman19:10:47

if you are just using an int as a handy bitvector it doesn't matter

strickinato20:10:26

Hi - I’m looking for a bit of help. Question in thread so as not to derail current conversation

strickinato20:10:31

Hey - all, I’m looking for a suggestion for how to deal with a Date issue I’m dealing with - regarding how clj-time turns things into timestamps. My goal is to save a DATE in postgres: I’m using the postgres driver for jdbc and clj time. Fundamentally - it appears my problems are arising from the fact that I want to use “Dates” instead of DateTimes. Here’s a minimal version of the problem: Try to save Oct. 10, 1980:

(-> "1980-10-10"
      (clj-time.coerce/to-sql-date) ;; => #inst "1980-10-10T00:00:00.000-00:00"
      (#(clojure.java.jdbc/update! db/url :friends {:birthday % } ["id = 1"]))
      )
It appears OK and clojure is happy to persist this. But then: SELECT birthday FROM friends WHERE id = 1; => 1980-10-09 It saves as the 9th! I believe this issue is because I’m west of UTC- so it’s saving a timestamp for midnight UTC of the 10th, which is the 9th in my time zone - but I can’t figure out how to actually get it to properly save without adding an arbitrary amount of time - which feels really wrong. In fact - the following works:
(-> "2000-10-10"
      (clf-time.coerce/from-string)
      (#(clj-time.core/plus % (time/hours 6)))
      (clj.time.coerce/to-sql-date) ;; => #inst "2000-10-10T06:00:00.000-00:00"
      (#(clojure.java.jdbc/update! db/url :friends {:birthday % } ["id = 1"]))
      )
Any thoughts or suggestions?

gon07:10:38

An option could be having your db column's type with NO TIMEZONE date type

Denis G20:10:49

Why there is so much hype about go-routines? Don’t know how go-routines in Clojure are similar to Go’s ones, but everybody is so excited of “cheap” threads. What are they? Just green threads (used from Thread Pool) that operate on cooperative scheduling? How can you make threads with smaller stack than the OS ones? They still need to use OS Threads right?

andy.fingerhut20:10:37

@mfiano If you want to pass around 64-bit unsigned ints, you can put their bits into a 64-bit Java long, and just don't use < > etc. operations on it. bits are bits.

hiredman20:10:16

@denisgrebennicov of course when the code is executing it has to be executing on some os thread, so so there is no savings there, but a lot of the time code is executing, the thread is stopped waiting on some condition, or some io or some timeout. If you can capture what the thread would start doing when it resumed (the continuation) you push that aside somewhere to start again once whatever condition the thread was waiting for happens, then reuse that same thread to execute something else

hiredman20:10:37

so you could have a million light weight threads (green threads, go routines, logical threads) that stop and wait for something a lot, so they can all be multiplexed running on a single os thread

hiredman20:10:38

that is at least the theory.

Denis G20:10:56

@hiredman but when thread is waiting for some io, can not we just call Thread:yield?

credulous20:10:46

Hi! I already posted this on IRC, but I have a feeling slack might get a bit more traffic? I’m having trouble accessing a native JNI lib from my Clojure project. I can run sample code in Java that access the lib and wrapper .class files OK, but I can’t load it into the repl.

credulous20:10:54

I followed the instructions on the web page for the project I’m accessing (CoolProp), which just meant putting the .jnilib (mac OSX) file in the project directory and all the wrapper .java files in random subdirectory, then compiling them all.

credulous20:10:33

the sample code just loads the native library and accesses static methods:

credulous20:10:40

public class Example {
     static {
       try {
         System.loadLibrary("CoolProp");
       } catch(UnsatisfiedLinkError e) {
         e.printStackTrace();
       }
     }

     public static void main(String argv[]){
         System.out.println("**************** INFORMATION ***************");
         System.out.println("This example was auto-generated by the language-agnostic dev/scripts/example_generator.py script written by Ian   Bell");
         System.out.println("CoolProp version:" + " " + CoolProp.get_global_param_string("version"));

credulous20:10:58

That works fine. But I don’t know how to tell my Clojure project to load it. The files don’t have a package. I tried (import '(CoolProp)) but that failed.

hiredman20:10:36

@denisgrebennicov Thread.yield does nothing

noisesmith20:10:12

@denisgrebennicov an in-process context switch (green threads) has much lower overhead than an OS level context switch. It can make informed decisions about the context switch that an OS simply can't do generally, and it doesn't have to do the same kind of isolation.

👍 4
hiredman20:10:48

but yeah, I think you go go pretty far on a modern jvm on a modern linux system with the limits on the number of threads bumped up without needing anything like core.async

Denis G20:10:49

@hiredman Thread yield indeed voluntarily gives up the time the thread has. Meaning no busy-waiting is necessary. AFAIK

Denis G20:10:13

not on JVM or in general?

hiredman20:10:38

both basically

hiredman20:10:54

some of that is just semantics, because thread yield doesn't "give up" the thread

Denis G20:10:05

@noisesmith do green threads switch greedily? Meaning one runs as long as it cans or is there a time-limit?

noisesmith20:10:42

they require explicitly yielding somehow (cooperative multithreading) in the instances I know of (in core.async this is done by parking operations)

hiredman20:10:57

it may give up the thread's scheduling spot so the os scheduler can schedule another thread, which if you wave your hands and kind of squint at it is kind of the same thing

hiredman20:10:55

but it doesn't keep the same thread (stack, whatever os bookkeeping structures exist, etc) and execute different code on the same thread

hiredman20:10:29

but also on modern jvms I am pretty sure Thread.yield is close to a noop

Denis G21:10:26

gotcha. Thanks 🙂 Getting smarter feels good 😅

awb9921:10:12

I want to get started with Paredit or ParInfer in InteliJ/Cursive. Does anyone know of good tutorials for those? The best thing I found is the ParInfer website. For ParEdit I found pretty much nothing. Also, what woudl you recommend? Paredit or ParInfer ?

jstaab21:10:57

Anyone have any suggestions for a library that would let me create a directed acyclic graph of functions, then on invoking one with context, call all dependencies and provide the results to the called function, memoizing called functions to avoid tons of work? I was looking at integrant, but it doesn't seem like quite the right fit for this use case. So for example:

(defnode get-n
  {:call (fn [ctx deps idx] (->> ctx :n-list (nth idx)))
   :deps []})

(defnode add-x-n
  {:call (fn [ctx deps m] (+ (:n deps) m))
   :deps
   [{:node get-n
     :name :n
     :args (fn [ctx m] [(-> ctx :x :n-idx)])}]})

(add-x-n {:n-list [3 5 8] :x {:n-idx 0}} 10) ;; 13
Obviously something that implemented references across the data structure, and handled collections would be cool.

hiredman21:10:26

I have the gist for you

awb9921:10:17

@hiredman Amazing! you have solutions for everything 🙂

hiredman21:10:05

(happens to be built on list processing, but could do individual values)

jstaab22:10:02

My brain is exploding right now

jstaab22:10:22

So that's basically like a poor man's logic engine? I'm looking less for a combinatorial expansion than a way to do pull-based (reading this article right now: https://medium.com/@kumarshantanu/dependency-injection-with-clojure-using-dime-af57b140bd3f) lazy calculations to figure out a single result.

jstaab22:10:52

Hmm although I guess it need not expand the result set...

hiredman22:10:34

no, it isn't a logic engine, it just happens to be built on mapcat, so it builds seqs of answers and mapcats the dependent functions over them

hiredman22:10:23

the key is the topological sorting function, once you turn the graph in to a linear set of nodes to visit, that kind of problem is just a reduce

jstaab22:10:03

interesting

hiredman22:10:46

(defn run [g]
  (->> (zipmap (keys g)
               (for [v (vals g)]
                 (set (map (comp keyword name) (:deps (meta v))))))
       topo
       (reduce (fn [m k]
                 (let [f (get g k)] (assoc m k ((if (fn? f) f (constantly f)) m)))) {})))

(run {:a (f [] 10)
       :b (f [a a] (- a a))
       :c (f [a] (inc a))
       :d (f [b c]
             (+ b c))})

;=>  {:a 10, :b 0, :c 11, :d 11}

jstaab22:10:51

@borkdude yeah, it is a similar scenario; deeply nested values that depend all over the place

hiredman22:10:37

so that is a much simpler version of run that doesn't try to get fancy with seq processing

jstaab22:10:14

Nice, ok. The seq processing might actually come in handy for my use case

hiredman22:10:56

run is sort of a tool for composing functions, so it can be parameterized over how you apply functions, so for example function application for a monad is whatever the monadic bind operation is, so you can get different versions of run for different monads (seqs, futures, identity, etc)

hiredman22:10:48

always specify timezones

hiredman22:10:20

my guess is your database is configured for a different default timezone from your jvm

hiredman22:10:40

and the type you are using in postgres is timezoneless

hiredman22:10:15

and I think the postgres jdbc driver makes some assumptions about dates and tries to convert them, but it is kind of stupid about it and it will break things

hiredman22:10:31

so either 1. specify the default timezone every where for databases, runtimes, etc as utc or 2. make timezones explicit

hiredman22:10:53

ah, I take that back, I don't think the jdbc driver is being stupid about conversion, what it is is when constructing a date object on the jvm you need to specify a timezone, and if the underlying sql type doesn't have one the driver defaults to jdbc

strickinato22:10:49

Interesting - thanks for the response. > specify the default timezone every where for databases, runtimes, etc as utc I was hoping to not have to do this 😞 - it’s weird to represent days as a moment in time 😞 > when constructing a date object on the jvm you need to specify a timezone I suppose this caught me off guard. There is a Date object distinct from a DateTime one- which I think I took to assume would mean that you could represent Date’s independent of time zones, but I guess not

hiredman22:10:16

dates need timezones too, it is silly that postgres doesn't support a date with a timezone