Fork me on GitHub
#off-topic
<
2017-06-19
>
gmercer08:06:00

@qqq in mysql - schema may help for grouping

gmercer08:06:46

@qqq INSERT IGNORE may help with insert or update

yonatanel13:06:03

@qqq re MySQL https://dev.mysql.com/doc/refman/5.7/en/insert-on-duplicate.html. (Sorry for answering this old question, it looked like the latest one in slack but now I got the rest...)

dominicm16:06:38

Slack's UI blocks on notification sends. So if your notification agent locks up, slack will silently lock up every time someone pings you. That's been a fun few hours finding out.

dominicm17:06:12

@qqq postgres also has schema. Tables are created into public schema by default.

naomarik19:06:16

i feel like i just blinked and grunt/gulp became deprecated to this confusing build tool

mobileink19:06:47

it's a curious thing, that clojurians often stress (immutable) data over functions. but afaics, an immutable datum is indistinguishable from a "pure" (nullary) function. so why stress data over functions? after all many clj data structures behave as functions, maps for example. "everything is a function" works a lot more smoothly than "some things are fns and some are data", no?

noisesmith19:06:58

but functions are opaque and easily have hidden side effects, sanely constructed data is transparent and lacks side effects

noisesmith19:06:30

clojure’s design tends to prefer explicit and transparent over implicit and hidden

noisesmith19:06:21

but really lazy-seqs are in a fuzzy place here, since someone can easily hide side effects in the generator, or it could just be very expensive resources wise to generate item n+1

mobileink19:06:38

"functions", by definition, have no side effects. theoretically, at least, there are no data, only functions (lambda calculus).

mobileink19:06:16

computational routines may have side-effects, but then they do not represent functions.

pesterhazy19:06:54

"functions", by definition, have no side effects. - that's only one (and not the most common) usage of the word "function" in programming

pesterhazy19:06:04

"a subroutine may be called a procedure, a function, a routine, a method, or a subprogram." (https://en.wikipedia.org/wiki/Subroutine)

mobileink19:06:36

well, people can use a word however they please, of course. but "function" is pretty well-defined.

pesterhazy19:06:54

it has different meanings in different contexts, and in the context of programming languages, I often use "function" and "subroutine" interchangably

mobileink19:06:11

obviously there are no (or few) true functions in a program. but we can and do entertain the fiction which helps us reason.

mobileink19:06:48

then how do you distinguish between routines that emulate fns (no side effects) and those that do not? anyway my point is about the elevation of "data".

pesterhazy19:06:20

ever heard of pure functions? 🙂

mobileink19:06:26

sure. ever heard of the department of redundancy department? 😉

mobileink19:06:48

if its a fn, it's pure.

pesterhazy19:06:30

anyway, I also found it interesting that in some programming languages (like elm, maybe haskell?), vars are simply 0-arity functions

mobileink19:06:18

yeah, sounds like what i'm thinking of. is '2' a datum or a nullary fn?

mobileink19:06:09

it's all perspective, i reckon

pesterhazy19:06:11

in those languages they're also used ("invoked") the same way as functions, whereas in lisp or C-like languages, application must be requested explicitly (using parentheses)

pesterhazy19:06:28

but clojure's point is that data is better because it's transparent: it can be serialized, it wears its nature on its sleeve

mobileink19:06:05

fwiw i've been trying (again!) to grok category theory. there, you can discard objects (data?) in favor of arrows (fns?)

mobileink19:06:07

but fns also wear there natures on their sleeves. there's no diff, that i can see.

pesterhazy19:06:58

how do you iterate over a function? how do you transform a function?

mobileink19:06:03

the fn '+' is just as transparent as '2', no?

pesterhazy19:06:32

'(1 2 3) can be manipulated more directly than (fn [] (cons 3 (cons 2 (cons 1 nil))))

mobileink19:06:02

you iterate over a fn by applying it to args. e.g. a vector.

noisesmith19:06:03

in the vm we have available, introspecting a fn is sloppy and error prone

noisesmith19:06:17

in that same vm, introspecting data is straightforward

mobileink19:06:14

@pesterhazy that's just syntax. the result is the same.

noisesmith19:06:55

@mobileink it’s not just syntax - the type of (fn …) is just a class, and you can’t walk its internals - you have to make it return somethign, unless you want to be an oracle and interpret its bytecode

bbqbaron19:06:55

@mobileink could you help me clarify my thinking by distinguishing what you’re talking about from macros, which are the treatment of a representation of code as if it were data?

pesterhazy20:06:15

wow I just saw that @noisesmith made my point already, sorry for reiterating without attributing

mobileink20:06:38

@bbqbaron macros traffic in syntax. that's code, but neither fn nor data.

mobileink20:06:39

@noisesmith they both must be evaluated before you can do anything with them. adding fn just rearranges things.

bbqbaron20:06:48

i understand. i guess what i don’t understand is what representation of a function we are treating as data. some notion of its internal logic? but a representation of that logic that is neither a. whatever incidental bytecode results or b. syntax?

mobileink20:06:38

take maps or vectors as an example. usually treated as data. but i maintain it's better to think of them as fns - which clojure does.

noisesmith20:06:12

but I can check for keys in a map easier than I can check for which arguments a function accepts

noisesmith20:06:32

our functions don’t reveal themselves the way our data does

mobileink20:06:23

that's an accident of the language, i think.

bbqbaron20:06:42

what if a map could hold keys representing predicates? could there be a way for a map to be a transparent cond of some kind?

bbqbaron20:06:48

is there a way to bleed the distinction?

noisesmith20:06:52

it’s not an accident, it’s fundamental to the language’s model, and the model you want would require writing a new vm

noisesmith20:06:13

it wouldn’t allow using the host vm like we do now (if we care about efficiency like, at all)

bbqbaron20:06:25

(sorry, i don’t mean “in clojure right now”; i mean conceptually)

mobileink20:06:11

clojure could easily provide an 'args' fn that would work just like 'keys'.

mobileink20:06:18

@bbqbaron not sure i follow. the keys of a map can be fns (predicates).

bbqbaron20:06:44

they can? i had no idea. i guess it had never occurred to me to try that

bbqbaron20:06:21

huh; so they can!

mobileink20:06:22

yeah, you can use just about anything as a map key. how cool izzat?

mobileink20:06:49

pt being a map is a fn, and vice-versa.

noisesmith20:06:58

but a map is “typed” in a way that a function is not - you can introspect and see exactly the args it responds to

noisesmith20:06:10

not just the types of args, but specific arg values

bbqbaron20:06:25

yes, definitely; the case i was asking about would require some first-class notion that the map is keyed by a subset of functions

mobileink20:06:06

@pesterhazy , @noisesmith we can also write (fn [] (fn [] .... )), but so what?

mobileink20:06:51

@noisesmith what do you mean by "introspect"? keys?

noisesmith20:06:50

or anything else that lets me look at the structure contained inside the map

noisesmith20:06:13

functions as we know them have implicit structure that is a side effect of their bytecode as it acts on an input

noisesmith20:06:30

a hash-map has an explicit well known structure

mobileink20:06:15

so does every fn. (keys +) => Number. clojure does not support this but it could.

noisesmith20:06:35

I’d be interested to see how the language you are suggesting would work, but it seems very strange to me

noisesmith20:06:49

even in the languages I know of that have inspectable functions (unlike clojure), they aren’t inspectable in a declarative form one can easily reason about

mobileink20:06:39

the point i'm failing to make is that there is no prinicipled reason to privilege data over fns, conceptually. the stuff we've been talking about is mostly implementation detail.

mobileink20:06:24

or, the diff between a simple clojure map qua fn and a big hairy defn is one of degree, not kind.

dpsutton20:06:43

does anyone use elasticsearch? i'm thinking of adding logging from some applications and the indexing from a .NET application seems really slow

dpsutton21:06:59

wondering how you use it if anyone has experience

bbqbaron21:06:36

from .NET per se? ES indexing always seemed much slower than searching, to me

bbqbaron21:06:51

there are some internals you can manipulate, like the size of the internal bulk-indexing queue

dpsutton21:06:54

just in general

bbqbaron21:06:59

(if you’re administering your own)

dpsutton21:06:06

we will be. just getting into it

dpsutton21:06:23

do you use outside processes to send things (like logstash) or do you index from your applications?

bbqbaron21:06:36

from the apps; it’s a source of business data for us

bbqbaron21:06:47

mostly JVM, some JS lambdas

dpsutton21:06:13

ok. it just seems really slow in some initial tests i had.

dpsutton21:06:27

how much data are you putting into it?

bbqbaron21:06:33

~100M docs?

dpsutton21:06:44

per what kind of unit timeframe?

bbqbaron21:06:38

usually over a pretty long period--or rather we could afford to wait. rare that we’ve had to write all of them cold. if we did, the bottleneck would probably be calculation, not indexing, since they’re the result of a fairly compute-intensive process

bbqbaron21:06:03

if we had to outright replace the index from known good data, i think it took…8 hours? but that’s highly anecdotal/secondhand

bbqbaron21:06:31

apologies if you already know any of these other points, but they were new to us, so in case they help: * ES loves bulk writes instead of many small requests * docs cannot be partially updated; they’re always outright replaced, even if using things like the groovy API to perform updates “in place”

dpsutton21:06:07

thanks for the info

dpsutton21:06:23

company sent me to training but it focuses on the querying part and not how to get it in there