Fork me on GitHub
#clojure
<
2019-05-04
>
duncanmak15:05:46

I just wanna say, I’m really happy that clojure.pprint/cl-format exists, it’s a cool piece of work!

jumar16:05:12

Cool, didn't know that such a thing exists. But be aware of https://dev.clojure.org/jira/browse/CLJ-1950

Takis_17:05:50

hello,i am reading a spark book,and i am trying it with Clojure,everything is simple so far,i also saw the sparkling library,the question i have is this

Takis_17:05:28

i can wrap select of spark to make the use more simple like this

Takis_17:05:29

(defn select [dataset & col-names] (.select ^Dataset dataset (into-array (map (fn [c] (if (string? c) (functions/col c) c)) col-names))))

Takis_17:05:00

but when i use it,catalyst knows what my select function is? or i loss the optimizations?

Takis_17:05:16

in general i was wondering if using spark from Clojure is simple or problems will occur ,but so far it seems simple to use spark from Clojure

Takis_18:05:48

i found that the plans are exactly the same,so spark i guess has the way to find what my function will do to the dataframe thanks

respatialized19:05:16

Yeah it's just a declarative specification either way, regardless of what intermediate functions you call to get there. Catalyst is pretty smart.

todo20:05:47

(not trolling, geniune question) Is there a useable GADT (like subset of ocaml, subset of rust) type system for use in Clojure yet, or is Racket the closest to "scheme with gadt" ?

seancorfield20:05:34

@todo I'm not sure what GADT would even look like in Clojure.

todo20:05:00

I'd settle for just sum + union types

todo20:05:45

to be able to declare things like (struct Cat (:name String) (:age int) (:lives_left int)) (struct Dog (:name String) (:age int) (:can-defend bool)) (enum Animal [Cat, Dog])

todo20:05:08

mainly I want to be able to look at a var and know valid fields on the var (rather than explore it in a REPL)

seancorfield20:05:27

Have you looked at clojure.spec?

todo20:05:40

Yes, spec is not what I want.

seancorfield20:05:13

Clojure is a dynamically typed language. What you're asking for doesn't really fit in.

didibus20:05:29

Doesn't Gadt not make any sense in a dynamically typed environment?

todo20:05:34

Specs provides guarantee of "these invariants seem to hold on these randomized tests", whereas I'm looking for "the compiler will not allow these invaraints to be broken"

vemv21:05:32

Keep an eye on https://github.com/arohner/spectrum Meanwhile you can use sth like https://github.com/jeaye/orchestra . If you spec code (as opposed to data*), useful errors will pop up far more frequently than with generative testing alone Basically it's like Clojure's :pre and :post *non exclusive approaches

seancorfield20:05:35

clojure.spec, multi-methods. Those are Clojure's tools for solving that sort of problem.

seancorfield20:05:52

Or records and protocols.

seancorfield20:05:32

@todo Have you looked at "Typed Clojure" -- yeah, was just about to suggest that...

didibus20:05:38

Are you saying you want a DSL within Clojure which has static type guarantees of the kind that also supports GADT?

todo20:05:37

@didibus: DSL or host language -- I'm okay with either. I'm looking for a practical scheme that supports sum + union types.

todo20:05:52

@seancorfield: is typed.clojure used in practice? I thought it was still experimental

seancorfield20:05:41

@todo Typed Clojure is an ongoing experiment that Ambrose has been working on for years. We looked at using it at work years ago but it was difficult to use -- and Ambrose has done a lot of work on making it more gradual and easier to use.

didibus20:05:28

Not quite GADTs though

seancorfield20:05:51

We tried Schema and Typed Clojure several times, and went back and forth between them, and gave up on both in the end. We are heavy users of clojure.spec at work tho'...

ghadi20:05:06

I don't care about compile time vs runtime errors. The salient part is I catch the errors "before it leaves my laptop". spec + test.check give me that

didibus20:05:46

Otherwise, there's very little apart from core.typed for static type checking in Clojure. That said, all types are union and sum of everything all the time. So you can still model things that way easily. You just don't get compiler checks.

ghadi20:05:56

It's like going to an Italian restaurant and asking for gluten free... Gotta embrace the fact that Clojure is a dynamic language

seancorfield20:05:19

Hey, in California, you can get gluten free in Italian restaurants! 🙂

🇮🇹 4
didibus20:05:35

If you want real typed lisp, I'd look at either carp.lang or typed racket or shen

ghadi20:05:42

Hahaha I always forget about Cali @seancorfield

🐻 4
todo20:05:52

@ghadi I think it's more like walking into a Mac store and asking "do you have windows emu support yet" ?

didibus21:05:59

I think the thing is, Lisp don't work very well with strong static types. Because fundamentally, they're about interactive inside out development.

didibus21:05:59

For example, appart for carp.lang, I don't think I ever saw a static typed lisp, whose static typing wasn't gradual or optional.

vemv21:05:32

Keep an eye on https://github.com/arohner/spectrum Meanwhile you can use sth like https://github.com/jeaye/orchestra . If you spec code (as opposed to data*), useful errors will pop up far more frequently than with generative testing alone Basically it's like Clojure's :pre and :post *non exclusive approaches

dpsutton21:05:52

i'm going through Programming Languages Application and Interpretation and its in a typed lisp. They exist

dpsutton21:05:13

there's also shen i think?

dominicm22:05:30

Racket was a cited example above

didibus22:05:33

Racket and Shen are both optionally/gradually typed only.

didibus22:05:48

The thing is, the static types restrict what you can do. And Lisps want to do things that don't bode well with static types. Most of core.typed was issues with trying to figure out ways to type existing core functions. Similarly, the syntax doesn't work very well with types either. Where do you add the type defs in the sexpr?

seancorfield22:05:27

Yeah, when we used core.typed, one of the biggest problems I had was getting it to accept code that was idiomatic but hard to infer types from -- so I found myself refactoring to much less idiomatic structure in order to successfully type check the code.

didibus22:05:54

There's very simple examples (and maybe someone with better type theoretic knowledge can correct me), but for example, I have a function that takes a keyword followed by one or more values until another keyword is reached which can also be followed by one or more values. What type is that?

didibus22:05:30

[:add 10 23 45 66 83 :remove 23 98 34]

didibus22:05:33

For example

didibus22:05:08

You could say it's a vector of either keywords or ints

seancorfield22:05:36

clojure.spec can specify that. I suspect dependent types (like in Coq etc) can as well?

didibus22:05:18

But its not really what it is. So that would be pretty loose, and almost worthless for correctness. Since bugs would be things like using the wrong keyword, or having the order of the keywords wrong.

didibus22:05:24

@seancorfield Ya exactly, that's where I was getting at with my example. Spec is much better. Because I can spec those highest level constraints, like that :add should come before :remove for example.

didibus22:05:15

Maybe dependent typed could as well. But it's my in impression they still require too much effort from the programmer for widespread adoption. Not a lot of people are willing to take the productivity hit from using them

seancorfield22:05:25

Yeah. Types get you some of the way but no matter how much effort you put in, they can't specify as much as clojure.spec in some areas -- and the same is true in the other direction. Types give you an upfront safety net that certain incorrect code won't compile, which can save time (but of course, certain correct code won't compile either).

didibus22:05:13

For sure. I just feel overall, the things people like about Lisps are the things that are hard to impossible to type, starting with heterogeneous lists of variable length. You can cons anything on them, and that's amazing. If you constrain what you can cons on it... well, such a Lisp is slowly becoming less of a Lisp and more of just Haskell

didibus22:05:55

So you reach a point where there is no point in having strong static typed Lisp. And you realize if that's what you want, just use Haskell or OCaml or Java even

seancorfield22:05:48

Aye, I've used everything from assembly up to Haskell and I'm one of those devs who just gets frustrated with type systems. I think programmers are inherently either "statically typed people" or "dynamically typed people".