Contents^

Table of Contents
date title user score
2019-11-14 04:01:54 Show HN: Bamboolib – A GUI for Pandas (Python Data Science) __tobals__ 13
2019-11-09 09:26:55 Tools for turning descriptions into diagrams: text-to-picture resources ingve 61
2019-10-16 00:42:33 CSR: Corporate Social Responsibility westurner 2
2019-10-19 08:28:01 GTD Tickler file – a proposal for text file format vivekv 3
2019-10-20 02:07:48 Ask HN: Any suggestion on how to test CLI applications? pdappollonio 3
2019-10-16 00:34:32 The Golden Butterfly and the All Weather Portfolio westurner 1
2019-10-12 07:19:23 Canada's Decision To Make Public More Clinical Trial Data Puts Pressure On FDA pseudolus 192
2019-10-10 23:35:35 Python Alternative to Docker gilad 3
2019-10-09 00:17:45 $6B United Nations Agency Launches Bitcoin, Ethereum Crypto Fund zed88 8
2019-10-08 16:03:02 Timsort, the Python sorting algorithm alexchamberlain 407
2019-10-07 22:29:21 Supreme Court allows blind people to sue retailers if websites aren't accessible justadudeama 743
2019-10-04 11:15:12 Streamlit: Turn a Python script into an interactive data analysis tool danicgross 467
2019-09-23 16:43:51 Scott’s Supreme Quantum Supremacy FAQ xmmrm 600
2019-09-23 18:31:40 Ask HN: How do you handle/maintain local Python environments? PascLeRasc 103
2019-09-23 12:35:51 Is the era of the $100 graphing calculator coming to an end? prostoalex 361
2019-09-23 03:17:17 Reinventing Home Directories Schiphol 118
2019-09-23 03:00:38 Serverless: slower and more expensive kiyanwang 1787
2019-09-22 17:32:04 Entropy can be used to understand systems acgan 3
2019-09-18 07:24:36 New Query Language for Graph Databases to Become International Standard Anon84 290
2019-09-21 13:21:03 A Python Interpreter Written in Python nnnmnten 2
2019-09-21 11:51:00 Reinventing Home Directories – systemd-homed [pdf] signa11 3
2019-09-21 13:08:28 Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM unbalancedparen 585
2019-09-19 20:00:14 Craftsmanship–The Alternative to the 4 Hour Work Week oglowo3 4
2019-09-19 09:31:43 Solar and Wind Power So Cheap They’re Outgrowing Subsidies ph0rque 623
2019-09-18 06:52:46 Show HN: Python Tests That Write Themselves timothycrosley 131
2019-09-09 10:52:49 Most Americans see catastrophic weather events worsening elorant 102
2019-09-17 12:00:54 Emergent Tool Use from Multi-Agent Interaction gdb 332
2019-09-17 22:32:25 Inkscape 1.0 Beta 1 nkoren 603
2019-09-08 13:45:57 Where Dollar Bills Come From danso 69
2019-09-05 07:13:24 Monetary Policy Is the Root Cause of the Millennials’ Struggle joshuafkon 52
2019-08-30 15:42:12 Non-root containers, Kubernetes CVE-2019-11245 and why you should care zelivans 8
2019-08-25 23:49:46 How do black holes destroy information and why is that a problem? sohkamyung 195
2019-08-25 09:48:11 Banned C standard library functions in Git source code susam 502
2019-08-25 10:01:30 Ask HN: What's the hardest thing to secure in a web-app? juansgaitan 7
2019-08-22 01:29:43 Crystal growers who sparked a revolution in graphene electronics sohkamyung 85
2019-08-22 16:27:43 Things to Know About GNU Readline matt_d 204
2019-08-22 16:16:41 Show HN: Termpage – Build a webpage that behaves like a terminal brisky 5
2019-08-21 22:49:19 Vimer - Avoid multiple instances of GVim with gvim –remote[-tab]-silent wrapper grepgeek 6
2019-08-22 16:06:27 Electric Dump Truck Produces More Energy Than It Uses mreome 3
2019-08-21 17:34:53 Ask HN: Let's make an open source/free SaaS platform to tackle school forms busymichael 12
2019-08-21 14:18:17 Ask HN: Is there a CRUD front end for databases (especially SQLite)? Tomte 2
2019-08-20 06:43:31 California approves solar-powered EV charging network and electric school buses elorant 15
2019-08-17 10:58:03 You May Be Better Off Picking Stocks at Random, Study Finds Vaslo 146
2019-08-12 08:15:23 Root: CERN's scientific data analysis framework for C++ z3phyr 137
2019-08-13 02:09:30 MesaPy: A Memory-Safe Python Implementation based on PyPy (2018) ospider 119
2019-08-11 16:22:30 Ask HN: Configuration Management for Personal Computer? jacquesm 197
2019-08-08 13:11:06 GitHub Actions now supports CI/CD, free for public repositories dstaheli 680
2019-08-05 17:19:30 The Fed is getting into the Real-Time payments business apo 96
2019-07-08 15:26:38 A Giant Asteroid of Gold Won’t Make Us Richer pseudolus 92
2019-07-08 10:52:06 Abusing the PHP Query String Parser to Bypass IDS, IPS, and WAF lelf 92
2019-06-28 14:23:33 Ask HN: Scripts/commands for extracting URL article text? (links -dump but) WCityMike 1
2019-07-02 11:02:08 NPR's Guide to Hypothesis-Driven Design for Editorial Projects danso 101
2019-06-20 14:56:56 Gryphon: An open-source framework for algorithmic trading in cryptocurrency reso 236
2019-06-21 00:18:36 Wind-Powered Car Travels Downwind Faster Than the Wind J253 5
2019-06-13 19:39:58 NOAA upgrades the U.S. global weather forecast model mehrdadn 214
2019-06-12 08:16:17 A plan to change how Harvard teaches economics carlosgg 116
2019-06-12 17:41:58 The New York Times course to teach its reporters data skills is now open-source espeed 423
2019-06-11 10:21:59 No Kings: How Do You Make Good Decisions Efficiently in a Flat Organization? eugenegamma 743
2019-06-01 23:13:28 4 Years of College, $0 in Debt: How Some Countries Make Education Affordable pseudolus 2
2019-05-26 10:16:10 Ask HN: What jobs can a software engineer take to tackle climate change? envfriendly 67
2019-05-23 12:59:05 YC's request for startups: Government 2.0 simonebrunozzi 194
2019-05-23 13:52:23 Almost 40% of Americans Would Struggle to Cover a $400 Emergency Geeek 112
2019-05-19 16:01:51 Congress should grow the Digital Services budget, it more than pays for itself rmason 68
2019-05-20 01:20:05 The Trillion-Dollar Annual Interest Payment westurner 2
2019-05-15 07:09:29 Oak, a Free and Open Certificate Transparency Log dankohn1 143
2019-05-14 09:36:21 Death rates from energy production per TWh peter_retief 122
2019-05-11 22:37:32 Use links not keys to represent relationships in APIs sarego 342
2019-05-09 23:49:28 No Python in Red Hat Linux 8? jandeboevrie 19
2019-05-06 09:16:47 JMAP: A modern, open email protocol okket 307
2019-05-09 14:51:33 Grid Optimization Competition zeristor 2
2019-05-02 16:11:54 Blockchain's present opportunity: data interchange standardization ivoras 2
2019-04-30 12:45:38 Ask HN: Value of “Shares of Stock options” when joining a startup cdeveloper 5
2019-04-28 13:46:48 CMU Computer Systems: Self-Grading Lab Assignments (2018) georgecmu 205
2019-04-28 14:50:29 Show HN: Debugging-Friendly Tracebacks for Python cknd 121
2019-04-28 07:41:27 Why isn't 1 a prime number? gpvos 273
2019-04-28 07:26:37 How do we know when we’ve fallen in love? (2016) rohmanhakim 157
2019-04-27 21:50:58 Rare and strange ICD-10 codes zdw 68
2019-04-20 15:10:14 Python Requests III maximilianroos 19
2019-04-17 09:43:04 Post-surgical deaths in Scotland drop by a third, attributed to a checklist fanf2 1036
2019-04-17 16:06:09 Apply to Y Combinator dlhntestuser 3
2019-04-02 03:51:50 Trunk-Based Development vs. Git Flow kiyanwang 4
2019-04-01 17:25:58 Ask HN: Anyone else write the commit message before they start coding? xkapastel 25
2019-03-27 03:29:30 Ask HN: Datalog as the only language for web programming, logic and database truth_seeker 21
2019-03-24 19:46:33 The cortex is a neural network of neural networks curtis 297
2019-03-22 21:51:49 Is there a program like codeacademy but for learning sysadmin? tayvz 7
2019-03-22 17:18:44 Maybe You Don't Need Kubernetes ra7 500
2019-03-21 08:04:34 Quantum Machine Appears to Defy Universe’s Push for Disorder biofox 78
2019-03-21 12:45:42 Pytype checks and infers types for your Python code mkesper 4
2019-03-20 21:56:26 How I'm able to take notes in mathematics lectures using LaTeX and Vim tambourine_man 674
2019-03-21 05:18:51 LHCb discovers matter-antimatter asymmetry in charm quarks rbanffy 269
2019-03-21 00:22:37 React Router v5 jsdev93 153
2019-03-15 18:23:21 Experimental rejection of observer-independence in the quantum world lisper 186
2019-03-15 08:14:22 Show HN: A simple Prolog Interpreter written in a few lines of Python 3 photon_lines 148
2019-03-07 17:57:28 How to earn your macroeconomics and finance white belt as a software developer andrenth 307
2019-03-02 14:24:35 Ask HN: Relationship between set theory and category theory fmihaila 4
2019-02-26 11:24:41 The most popular docker images each contain at least 30 vulnerabilities vinnyglennon 562
2019-02-24 22:39:39 Tinycoin: A small, horrible cryptocurrency in Python for educational purposes MrXOR 4
2019-02-20 14:08:47 When does the concept of equilibrium work in economics? dnetesn 54
2019-02-20 22:53:23 Simdjson – Parsing Gigabytes of JSON per Second cmsimike 597
2019-02-18 10:13:02 A faster, more efficient cryptocurrency salvadormon 583
2019-02-17 05:52:11 Git-signatures – Multiple PGP signatures for your commits Couto 75
2019-02-16 06:55:28 Running an LED in reverse could cool future computers ChrisGranger 46
2019-02-06 07:15:56 Compounding Knowledge golyi 481
2019-02-16 14:49:30 Why CISA Issued Our First Emergency Directive ca98am79 211
2019-02-14 23:22:11 Chrome will Soon Let You Share Links to a Specific Word or Sentence on a Page kumaranvpl 359
2019-02-09 12:21:30 Guidelines for keeping a laboratory notebook Tomte 87
2019-02-07 12:03:47 Superalgos and the Trading Singularity ciencias 2
2019-02-07 12:23:44 Crunching 200 years of stock, bond, currency and commodity data chollida1 308
2019-02-06 14:50:35 Show HN: React-Schemaorg: Strongly-Typed Schema.org JSON-LD for React Eyas 16
2019-02-06 16:15:33 Consumer Protection Bureau Aims to Roll Back Rules for Payday Lending pseudolus 197
2019-02-05 01:56:30 Lectures in Quantitative Economics as Python and Julia Notebooks westurner 355
2019-02-04 11:55:50 If Software Is Funded from a Public Source, Its Code Should Be Open Source jrepinc 1138
2019-02-04 23:55:48 Apache Arrow 0.12.0 westurner 1
2019-02-04 23:51:34 Statement on Status of the Consolidated Audit Trail (2018) westurner 1
2019-02-04 20:03:28 U.S. Federal District Court Declared Bitcoin as Legal Money obilgic 12
2019-01-30 12:42:06 Post Quantum Crypto Standardization Process – Second Round Candidates Announced dlgeek 2
2019-01-30 13:59:56 Ask HN: How do you evaluate security of OSS before importing? riyakhanna1983 5
2019-01-30 09:35:47 Ask HN: How can I use my programming skills to support nonprofit organizations? theneck 3
2019-01-29 19:43:16 Ask HN: Steps to forming a company? jxr006 4
2019-01-29 13:48:48 A Self-Learning, Modern Computer Science Curriculum hacknrk 394
2019-01-24 00:34:14 MVP Spec hyperpallium 2
2019-01-21 12:10:37 Can we merge Certificate Transparency with blockchain? fedotovcorp 3
2019-01-21 20:38:23 Why Don't People Use Formal Methods? pplonski86 419
2019-01-20 20:29:25 Steps to a clean dataset with Pandas NicoJuicy 4
2019-01-19 19:38:48 Reahl – A Python-only web framework kim0 165
2019-01-12 19:56:20 Ask HN: How can you save money while living on poverty level? ccdev 8
2019-01-11 14:46:52 A DNS hijacking wave is targeting companies at an almost unprecedented scale Elof 112
2019-01-09 23:09:59 Show HN: Generate dank mnemonic seed phrases in the terminal mofle 3
2019-01-08 15:28:29 Can you sign a quantum state? zdw 3
2019-01-09 18:04:41 Lattice Attacks Against Weak ECDSA Signatures in Cryptocurrencies [pdf] soohyung 11
2019-01-09 12:00:44 REMME – A blockchain-based protocol for issuing X.509 client certificates fedotovcorp 33
2019-01-08 09:51:20 California grid data is live – solar developers take note Osiris30 2
2019-01-05 12:30:30 Why attend predatory colleges in the US? azhenley 3
2018-12-31 15:43:54 Ask HN: Data analysis workflow? tucaz 1
2018-12-28 16:25:15 The U.S. is spending millions to solve mystery sonic attacks on diplomats johnshades 5
2018-12-27 10:00:38 Ask HN: What is your favorite open-source job scheduler bohinjc 6
2018-12-22 06:53:46 How to Version-Control Jupyter Notebooks tosh 164
2018-12-04 10:25:47 Teaching and Learning with Jupyter (A book by Jupyter for Education) westurner 5
2018-11-27 17:48:54 Margin Notes: Automatic code documentation with recorded examples from runtime mpweiher 67
2018-11-24 15:33:08 Time to break academic publishing's stranglehold on research joeyespo 692
2018-11-22 10:32:27 Ask HN: How can I learn to read mathematical notation? cursorial 211
2018-10-18 18:07:59 New law lets you defer capital gains taxes by investing in opportunity zones rmason 88
2018-10-15 19:55:06 How to Write a Technical Paper [pdf] boricensis 360
2018-10-15 15:19:40 JSON-LD 1.0: A JSON-Based Serialization for Linked Data geezerjay 2
2018-10-14 15:30:29 Jeff Hawkins Is Finally Ready to Explain His Brain Research tysone 489
2018-10-12 03:02:01 Interstellar Visitor Found to Be Unlike a Comet or an Asteroid Bootvis 204
2018-10-12 02:15:03 Publishing more data behind our reporting gballan 146
2018-10-10 22:23:44 CSV 1.1 – CSV Evolved (for Humans) polm23 84
2018-10-11 06:42:34 Ask HN: Which plants can be planted indoors and easily maintained? gymshoes 123
2018-10-08 10:23:38 Graduate Student Solves Quantum Verification Problem digital55 267
2018-10-05 07:53:30 The down side to wind power todd8 63
2018-10-05 05:47:19 Thermodynamics of Computation Wiki westurner 2
2018-10-04 09:27:48 Why Do Computers Use So Much Energy? tshannon 220
2018-09-30 22:11:07 Justice Department Sues to Stop California Net Neutrality Law jonburs 201
2018-09-22 10:52:45 White House Drafts Order to Probe Google, Facebook Practices Jerry2 105
2018-09-19 20:37:52 Ask HN: Books about applying the open source model to society kennu 1
2018-09-12 16:02:35 Today, Europe Lost The Internet. Now, We Fight Back DiabloD3 433
2018-09-01 14:13:52 Consumer science (a.k.a. home economics) as a college major guard0g 4
2018-08-28 11:18:26 Facebook vows to run on 100 percent renewable energy by 2020 TamoC 2
2018-08-30 12:51:10 California Moves to Require 100% Clean Electricity by 2045 dsr12 407
2018-08-29 11:15:59 Miami Will Be Underwater Soon. Its Drinking Water Could Go First hourislate 264
2018-08-29 22:50:51 Free hosting VPS for NGO project? vikramjb 1
2018-08-29 12:18:35 The Burden: Fossil Fuel, the Military and National Security westurner 3
2018-08-29 02:27:58 Scientists Warn the UN of Capitalism's Imminent Demise westurner 1
2018-08-28 14:41:52 Firefox Nightly Secure DNS Experimental Results Vinnl 40
2018-08-28 08:31:48 Long-sought decay of Higgs boson observed at CERN chmaynard 243
2018-08-28 09:00:54 Sen. Wyden Confirms Cell-Site Simulators Disrupt Emergency Calls DiabloD3 518
2018-08-23 00:01:34 Building a Model for Retirement Savings in Python koblenski 3
2018-08-20 21:38:10 New E.P.A. Rollback of Coal Pollution Regulations Takes a Major Step Forward yaseen-rob 3
2018-08-20 14:21:22 Researchers Build Room-Temp Quantum Transistor Using a Single Atom jonbaer 3
2018-08-20 10:55:17 New “Turning Tables” Technique Bypasses All Windows Kernel Mitigations yaseen-rob 2
2018-08-19 22:27:20 Um – Create your own man pages so you can remember how to do stuff quickthrower2 646
2018-08-15 04:52:10 Leverage Points: Places to Intervene in a System pjc50 113
2018-08-15 03:46:23 SQLite Release 3.25.0 adds support for window functions MarkusWinand 333
2018-08-15 19:53:03 Update on the Distrust of Symantec TLS Certificates dumpsterkid 3
2018-08-11 07:57:44 The Transport Layer Security (TLS) Protocol Version 1.3 dochtman 255
2018-08-12 08:56:52 Academic Torrents – Making 27TB of research data available jacquesm 1081
2018-08-10 15:19:24 1/0 = 0 ingve 650
2018-08-07 15:43:05 Power Worth Less Than Zero Spreads as Green Energy Floods the Grid bumholio 537
2018-08-05 15:27:39 Kernels, a free hosted Jupyter notebook environment with GPUs benhamner 95
2018-07-22 14:16:25 Solar and wind are coming. And the power sector isn’t ready spenrose 174
2018-07-11 13:15:47 Solar Just Hit a Record Low Price in the U.S toomuchtodo 456
2018-07-10 23:53:58 Causal Inference Book luu 104
2018-07-02 10:18:14 Tim Berners-Lee is working a platform designed to re-decentralize the web rapnie 36
2018-07-01 06:49:08 More States Opting to 'Robo-Grade' Student Essays by Computer happy-go-lucky 44
2018-07-02 07:26:28 Ask HN: Looking for a simple solution for building an online course r4victor 57
2018-06-30 15:45:56 There is now a backprop principle for deep learning on quantum computers GVQ 3
2018-06-30 21:03:36 New research a ‘breakthrough for large-scale discrete optimization’ new_guy 96
2018-06-29 23:17:31 Wind, solar farms produce 10% of US power in the first four months of 2018 toomuchtodo 85
2018-06-25 16:57:46 FDA approves first marijuana-derived drug and it may spark DEA rescheduling mikece 150
2018-06-21 10:22:43 States Can Require Internet Tax Collection, Supreme Court Rules uptown 541
2018-06-18 08:26:23 William Jennings Bryan’s “Cross of Gold” Speech zjacobi 71
2018-06-17 18:13:13 Ask HN: Do you consider yourself to be a good programmer? type0 27
2018-06-17 11:00:59 Handles are the better pointers ingve 194
2018-06-14 14:13:13 Neural scene representation and rendering johnmoberg 540
2018-06-17 20:19:20 New US Solar Record – 2.155 Cents per KWh prostoalex 4
2018-06-10 18:04:07 Ask HN: Is there a taxonomy of machine learning types? ljw1001 3
2018-05-22 16:22:43 Senator requests better https compliance at US Department of Defense [pdf] anigbrowl 168
2018-05-22 23:15:18 Banks Adopt Military-Style Tactics to Fight Cybercrime petethomas 3
2018-04-12 13:13:10 No, Section 230 Does Not Require Platforms to Be “Neutral” panarky 6
2018-04-11 14:28:06 Ask HN: Do battery costs justify “buy all sell all” over “net metering”? westurner 1
2018-04-09 21:17:43 Portugal electricity generation temporarily reaches 100% renewable mgdo 234
2018-04-06 19:16:25 GPU Prices Drop ~25% in March as Supply Normalizes merqurio 2
2018-04-09 23:51:08 Apple says it’s now powered by renewable energy worldwide iamspoilt 272
2018-03-18 13:13:15 Hackers Are So Fed Up with Twitter Bots They’re Hunting Them Down Themselves CrankyBear 271
2018-03-02 08:21:41 “We’re committing Twitter to increase the health and civility of conversation” dankohn1 147
2018-03-01 02:06:42 Gitflow – Animated in React v33ra 3
2018-02-28 22:06:35 Ask HN: How feasible is it to become proficient in several disciplines? diehunde 4
2018-02-27 09:47:40 After rising for 100 years, electricity demand is flat aaronbrethorst 629
2018-02-27 10:37:54 A framework for evaluating data scientist competency schaunwheeler 3
2018-02-27 18:28:01 Levi Strauss to use lasers instead of people to finish jeans e2e4 3
2018-02-27 18:24:45 Chaos Engineering: the history, principles, and practice austingunter 2
2018-02-27 09:52:39 Scientists use an atomic clock to measure the height of a mountain montrose 45
2018-02-27 18:10:10 Resources to learn project management best practices? chuie 1
2018-02-22 15:35:51 Ask HN: Thoughts on a website-embeddable, credential validating service? estroz 28
2018-02-21 05:03:58 Ask HN: What's the best algorithms and data structures online course? zabana 272
2018-02-20 15:14:40 Using Go as a scripting language in Linux neoasterisk 8
2018-02-18 12:09:07 Guidelines for enquiries regarding the regulatory framework for ICOs [pdf] paulsutter 23
2018-02-16 00:16:09 The Benjamin Franklin method for learning more from programming books nancyhua 566
2018-02-10 20:41:21 Avoiding blackouts with 100% renewable energy ramonvillasante 2
2018-02-10 11:25:54 Ask HN: What are some common abbreviations you use as a developer? yagamidev 3
2018-02-09 19:42:21 There Might Be No Way to Live Comfortably Without Also Ruining the Planet SirLJ 43
2018-02-08 22:52:44 Multiple GWAS finds 187 intelligence genes and role for neurogenesis/myelination gwern 2
2018-02-08 20:33:49 Could we solve blockchain scaling with terabyte-sized blocks? gwern 4
2018-02-07 20:50:24 Ask HN: Do you have ADD/ADHD? How do you manage it? vumgl 4
2018-02-03 14:36:02 Ask HN: How to understand the large codebase of an open-source project? maqbool 186
2018-02-03 13:56:30 What is the best way to learn to code from absolute scratch? eliotpeper 8
2018-02-02 04:35:58 Tesla racing series: Electric cars get the green light – Roadshow rbanffy 77
2018-02-02 13:40:19 What happens if you have too many jupyter notebooks? tvorogme 4
2018-02-01 00:49:46 Cancer ‘vaccine’ eliminates tumors in mice jv22222 942
2018-02-01 12:23:08 Boosting teeth’s healing ability by mobilizing stem cells in dental pulp digital55 306
2018-01-29 17:11:55 This Biodegradable Paper Donut Could Let Us Reforest the Planet westurner 2
2018-01-29 16:44:35 Drones that can plant 100k trees a day artsandsci 147
2018-01-27 22:21:28 What are some YouTube channels to progress into advanced levels of programming? altsyset 41
2018-01-25 17:41:24 Multiple issue and pull request templates clarkbw 17
2018-01-25 17:38:38 Five myths about Bitcoin’s energy use nvk 10
2018-01-23 18:41:16 Ask HN: Which programming language has the best documentation? siquick 3
2018-01-18 06:36:07 Ask HN: Recommended course/website/book to learn data structure and algorithms strikeX 3
2018-01-19 17:06:07 Why is quicksort better than other sorting algorithms in practice? isp 5
2018-01-18 16:16:16 ORDO: a modern alternative to X.509 juancampa 1
2018-01-18 11:47:03 Wine 3.0 Released etiam 724
2018-01-18 19:51:30 Kimbal Musk is leading a $25M mission to fix food in US schools rmason 2
2018-01-13 21:42:47 Spinzero – A Minimal Jupyter Notebook Theme neilpanchal 5
2018-01-11 13:27:17 What does the publishing industry bring to the Web? mpweiher 2
2018-01-10 14:02:09 Git is a blockchain Swizec 13
2018-01-07 12:06:03 Show HN: Convert Matlab/NumPy matrices to LaTeX tables tpaschalis 4
2018-01-02 10:48:10 A Year of Spaced Repetition Software in the Classroom misiti3780 4
2017-12-27 08:32:39 NIST Post-Quantum Cryptography Round 1 Submissions sohkamyung 130
2018-01-01 21:38:58 What are some good resources to learn about Quantum Computing? nmehta21 3
2017-12-29 15:53:06 Gridcoin: Rewarding Scientific Distributed Computing trueduke 134
2017-12-26 12:37:07 Power Prices Go Negative in Germany kwindla 485
2017-12-21 14:30:35 Mathematicians Find Wrinkle in Famed Fluid Equations digital55 240
2017-12-20 10:43:31 Bitcoin is an energy arbitrage js4 51
2017-12-19 17:03:30 There are now more than 200k pending Bitcoin transactions OyoKooN 192
2017-12-17 22:16:06 What ORMs have taught me: just learn SQL (2014) ausjke 540
2017-12-17 07:32:06 Show HN: An educational blockchain implementation in Python jre 412
2017-12-16 08:12:44 MSU Scholars Find $21T in Unauthorized Government Spending sillypuddy 137
2017-12-13 04:59:42 Universities spend millions on accessing results of publicly funded research versteegen 624
2017-12-11 19:49:44 An Interactive Introduction to Quantum Computing kevlened 254
2017-12-12 12:34:46 Quantum attacks on Bitcoin, and how to protect against them (ECDSA, SHA256) westurner 2
2017-12-10 17:50:44 Project Euler vinchuco 792
2017-12-12 10:17:39 Who’s Afraid of Bitcoin? The Futures Traders Going Short thisisit 54
2017-12-11 19:21:38 Statement on Cryptocurrencies and Initial Coin Offerings corbinpage 811
2017-12-11 15:02:04 Ask HN: How do you stay focused while programming/working? flipfloppity 83
2017-12-08 10:53:49 A Hacker Writes a Children's Book arthurjj 171
2017-12-11 18:17:52 Ask HN: Do ISPs have a legal obligation to not sell minors' web history anymore? westurner 2
2017-12-11 11:58:38 Tech luminaries call net neutrality vote an 'imminent threat' kjhughes 279
2017-12-06 18:55:25 Ask HN: Can hashes be replaced with optimization problems in blockchain? pacavaca 3
2017-12-01 01:19:43 Ask HN: What could we do with all the mining power of Bitcoin? Fold Protein? sova 3
2017-12-03 20:14:58 No CEO needed: These blockchain platforms will let ‘the crowd’ run startups maxwellnardi 4
2017-12-04 04:59:08 How much energy does Bitcoin mining really use? trueduke 3
2017-12-02 00:27:40 The Actual FCC Net Neutrality Repeal Document. TLDR: Read Pages 82-87 [pdf] croatoan 3
2017-12-01 21:55:26 The 5 most ridiculous things the FCC says in its new net neutrality propaganda pulisse 164
2017-12-01 13:15:47 FCC's Pai, addressing net neutrality rules, calls Twitter biased joeyespo 13
2017-12-01 05:49:25 A curated list of Chaos Engineering resources dastergon 51
2017-12-01 11:24:06 Technology behind Bitcoin could aid science, report says digital55 13
2017-11-30 15:07:26 Git hash function transition plan vszakats 215
2017-11-30 22:04:20 Vintage Cray Supercomputer Rolls Up to Auction ohjeez 3
2017-11-30 21:21:09 Google is officially 100% sun and wind powered – 3.0 gigawatts worth rippsu 163
2017-11-29 12:29:30 Interactive workflows for C++ with Jupyter SylvainCorlay 292
2017-11-28 16:01:32 Vanguard Founder Jack Bogle Says ‘Avoid Bitcoin Like the Plague’ dionmanu 105
2017-11-29 11:22:54 Nasdaq Plans to Introduce Bitcoin Futures knwang 416
2017-11-28 17:49:07 Ask HN: Where do you think Bitcoin will be by 2020? rblion 10
2017-11-28 18:03:11 Ask HN: Why would anyone share trading algorithms and compare by performance? westurner 1
2017-11-25 06:28:39 Ask HN: CS papers for software architecture and design? avrmav 513
2017-11-15 10:24:27 Keeping a Lab Notebook [pdf] Tomte 327
2017-10-28 08:12:53 How to teach technical concepts with cartoons Tomte 170
2017-10-22 16:43:03 Fact Checks fanf2 126
2017-10-19 05:51:13 DHS orders agencies to adopt DMARC email security puppetmaster30 2
2017-10-18 21:20:00 The electricity for 1BTC trade could power a house for a month niyikiza 25
2017-10-19 05:20:26 PAC Fundraising with Ethereum Contracts? westurner 1
2017-10-19 05:16:25 SolarWindow Completes Financing ($2.5m) westurner 2
2017-10-16 12:48:08 Here’s what you can do to protect yourself from the KRACK WiFi vulnerability tdrnd 2
2017-10-14 12:41:29 The Solar Garage Door – A Possible Alternative to the Emergency Generator curtis 2
2017-10-14 07:34:07 Using the Web Audio API to Make a Modem maaaats 307
2017-10-11 18:25:17 Ask HN: How to introduce someone to programming concepts during 12-hour drive? nkkollaw 9
2017-09-27 01:24:13 American Red Cross Asks for Ham Radio Operators for Puerto Rico Relief Effort kw71 346
2017-09-26 14:58:38 Technical and non-technical tips for rocking your coding interview duck 259
2017-09-23 12:12:36 Django 2.0 alpha orf 156
2017-09-24 00:15:28 Ask HN: What is the best way to spend my time as a 17-year-old who can code? jmeyer2k 161
2017-09-21 14:18:33 Democrats fight FCC's plans to redefine “broadband” from 25+ to 10+ Mbps gnicholas 18
2017-09-17 12:49:37 Ask HN: Any detailed explanation of computer science smithmayowa 2
2017-09-16 18:40:33 Ask HN: What algorithms should I research to code a conference scheduling app viertaxa 55
2017-09-15 05:51:45 What have been the greatest intellectual achievements? Gormisdomai 42
2017-09-15 23:22:02 Ask HN: What can't you do in Excel? (2017) danso 37
2017-09-08 20:04:36 Open Source Ruling Confirms Enforceability of Dual-Licensing and Breach of GPL t3f 116
2017-09-01 11:27:30 Elon Musk Describes What Great Communication Looks Like endswapper 90
2017-09-01 04:05:12 Great Ideas in Theoretical Computer Science tu7001 290
2017-08-28 16:06:24 Ask HN: How do you, as a developer, set measurable and actionable goals? humaninstrument 24
2017-08-26 16:06:24 Bitcoin Energy Consumption Index schwabacher 256
2017-08-26 09:59:19 Dancing can reverse the signs of aging in the brain brahmwg 71
2017-08-26 09:03:19 Rumours swell over new kind of gravitational-wave sighting indescions_2017 258
2017-08-20 12:56:37 New Discovery Simplifies Quantum Physics wolfgke 2
2017-08-23 03:22:00 OpenAI has developed new baseline tool for improving deep reinforcement learning grey_shirts 3
2017-08-24 23:19:03 The prior can generally only be understood in the context of the likelihood selimthegrim 94
2017-08-22 04:13:00 Ask HN: How to find/compare trading algorithms with Quantopian? westurner 3
2017-08-22 04:09:17 Ask HN: How do IPOs and ICOs help a business raise capital? westurner 2
2017-08-22 04:02:04 Solar Window coatings “outperform rooftop solar by 50-fold” westurner 4
2017-08-21 23:30:16 MS: Bitcoin mining uses as much electricity as 1M US homes pulisse 79
2017-08-15 15:45:47 Ask HN: What are your favorite entrepreneurship resources brianbreslin 13
2017-05-09 12:59:38 CPU Utilization is Wrong dmit 624
2017-05-06 17:13:03 Ask HN: Can I use convolutional neural networks to clasify videos on a CPU Faizann20 1
2017-05-01 10:17:36 Esoteric programming paradigms SlyShy 397
2017-04-27 04:41:09 gRPC-Web: Moving past REST+JSON towards type-safe Web APIs bestan 329
2017-04-16 03:59:55 Reasons blog posts can be of higher scientific quality than journal articles vixen99 233
2017-04-07 12:50:38 Fact Check now available in Google Search and News fouadmatin 302
2017-04-07 20:07:05 Ask HN: Is anyone working on CRISPR for happiness? arikr 4
2017-03-26 14:58:59 Roadmap to becoming a web developer in 2017 miguelarauj1o 4
2017-03-20 19:14:10 Beautiful Online SICP Dangeranger 762
2017-03-19 11:52:48 Ask HN: How do you keep track/save your learnings?(so that you can revisit them) mezod 4
2017-03-11 13:26:30 Ask HN: Criticisms of Bayesian statistics? muraiki 1
2017-01-16 18:53:09 80,000 Hours career plan worksheet BreakoutList 230
2017-01-07 18:27:31 World's first smartphone with a molecular sensor is coming in 2017 walterbell 19
2016-12-31 12:11:14 Ask HN: How would one build a business that only develops free software? anondon 12
2016-12-29 00:40:11 Ask HN: If your job involves continually importing CSVs, what industry is it? iamwil 12
2016-12-09 17:21:13 Ask HN: Maybe I kind of suck as a programmer – how do I supercharge my work? tastyface 328
2016-11-20 06:33:34 Ask HN: Anything Like Carl Sagan's Cosmos for Computer Science? leksak 32
2016-11-20 10:32:00 Learn X in Y minutes anonu 161
2016-11-03 05:46:50 Org mode 9.0 released Philipp__ 285
2016-11-13 00:23:33 Ask HN: Best Git workflow for small teams tmaly 166
2016-11-10 15:46:57 TDD Doesn't Work narfz 153
2016-11-07 14:13:48 C for Python programmers (2011) bogomipz 314
2016-10-26 02:19:06 Ask HN: How do you organise/integrate all the information in your life? tonteldoos 323
2016-10-23 14:06:00 Ask HN: What are the best web tools to build basic web apps as of October 2016? arikr 114
2016-10-16 10:55:18 Harvard and M.I.T. Are Sued Over Lack of Closed Captions lsh123 45
2016-10-06 11:15:16 Jack Dorsey Is Losing Control of Twitter miraj 283
2016-09-18 09:09:04 Schema.org: Mission, Project, Goal, Objective, Task westurner 49
2016-09-18 08:59:41 This week is #GlobalGoals week (and week of The World's Largest Lesson) westurner 1
2016-08-19 08:12:25 The Open Source Data Science Masters nns 95
2016-07-29 06:08:29 We Should Not Accept Scientific Results That Have Not Been Repeated dnetesn 910
2016-05-30 07:39:05 The SQL filter clause: selective aggregates MarkusWinand 138
2016-05-29 23:36:23 Ask HN: What do you think about the current education system? alejandrohacks 36
2016-05-10 08:55:01 A Reboot of the Legendary Physics Site ArXiv Could Shape Open Science tonybeltramelli 174
2014-03-23 14:27:04 Principles of good data analysis gjreda 108
2014-03-11 08:16:38 Why Puppet, Chef, Ansible aren't good enough iElectric2 362
2014-03-11 20:12:16 Python vs Julia – an example from machine learning ajtulloch 170
2014-02-17 10:23:21 Free static page hosting on Google App Engine in minutes fizerkhan 95
2014-02-03 09:15:30 “Don’t Reinvent the Wheel, Use a Framework” They All Say mogosselin 79
2013-09-09 10:20:50 IPython in Excel vj44 73
2013-08-11 01:56:12 PEP 450: Adding A Statistics Module To The Standard Library petsos 185
2013-08-02 21:03:51 Functional Programming with Python llambda 107
2013-08-01 10:59:55 PEP 8 Modernisation tristaneuan 213
2013-07-15 12:40:04 Useful Unix commands for data science gjreda 221
2013-07-13 11:35:40 The data visualization community needs its own Hacker News ejfox 11
2013-07-06 08:59:22 Ask HN: Intermediate Python learning resources? jesusx 113
2013-07-03 08:00:50 Ansible Simply Kicks Ass hunvreus 185
2013-06-29 05:44:08 Python-Based Tools for the Space Science Community neokya 76
2013-05-04 21:21:29 Debian 7.0 "Wheezy" released sciurus 428
2013-05-04 10:40:20 Big-O Algorithm Complexity Cheat Sheet ashleyblackmore 520
2013-05-03 22:32:14 JSON API steveklabnik 227
2013-05-04 14:04:39 Norton Ghost discontinued ruchirablog 42

Items^

[-]

Show HN: Bamboolib – A GUI for Pandas (Python Data Science)

This looks excellent. The ability to generate the Python code for the pandas dataframe transformations looks to be more useful than OpenRefine, TBH.

How much work would it be to use Dask (and Dask-ML) as a backend?

I see the OneHotEncoder button. Have you considered integration with Yellowbrick? They've probably already implemented a few of your near-future and someday roadmap items involving hyperparameter selection and model selection and visualization? https://www.scikit-yb.org/en/latest/

This video shows more of the advanced bamboolib features: https://youtu.be/I0a58h1OCcg

The live histogram rebinning looks useful. Recently I read about a 'shadowgram' / ~KDE approach with very many possible bin widths translucently overlaid in one chart. https://stats.stackexchange.com/questions/68999/how-to-smear...

Yellowbrick also has a bin width optimization visualization in yellowbrick.target.binning.BalancedBinningReference: https://www.scikit-yb.org/en/latest/api/target/binning.html

Great work.

CSR: Corporate Social Responsibility

> Proponents argue that corporations increase long-term profits by operating with a CSR perspective, while critics argue that CSR distracts from businesses' economic role.

... The 3 Pillars of Corporate Sustainability: Environmental, Social, Economic https://www.investopedia.com/articles/investing/100515/three...

Three dimensions of sustainability: (Environment (Society (Economy))) https://en.wikipedia.org/wiki/Sustainability#Three_dimension...

What are some of the corporate sustainability reporting standards?

How can I score a candidate portfolio with sustainability metrics in order to impact invest with maximum impact?

> What are some of the corporate sustainability reporting standards?

From https://en.wikipedia.org/wiki/Sustainability_reporting#Initi... :

>> Organizations can improve their sustainability performance by measuring (EthicalQuote (CEQ)), monitoring and reporting on it, helping them have a positive impact on society, the economy, and a sustainable future. The key drivers for the quality of sustainability reports are the guidelines of the Global Reporting Initiative (GRI),[3] (ACCA) award schemes or rankings. The GRI Sustainability Reporting Guidelines enable all organizations worldwide to assess their sustainability performance and disclose the results in a similar way to financial reporting.[4] The largest database of corporate sustainability reports can be found on the website of the United Nations Global Compact initiative.

The GRI (Global Reporting Initiative) Standards are now aligned with the UN Sustainable Development Goals (#GlobalGoals). https://en.wikipedia.org/wiki/Global_Reporting_Initiative

>> In 2017, 63 percent of the largest 100 companies (N100), and 75 percent of the Global Fortune 250 (G250) reported applying the GRI reporting framework.[3]

> How can I score a candidate portfolio with sustainability metrics in order to impact invest with maximum impact?

Does anybody have solutions for this? AFAIU, existing cleantech funds are more hand-picked than screened according to sustainability fundamentals.

[-]

GTD Tickler file – a proposal for text file format

Taskwarrior is also built upon the todo.txt format. [1]

Taskw supports various task dates – { due: scheduled: wait: until: recur: } [2]

Taskw supports various named dates like soq/eocq, som/eom (start/end of [current] quarter, start/end of month), tomorrow, later [3]

Taskw recurring tasks (recur:) use the duration syntax: weekly/wk/w, monthly/mo, quarterly/qtr, yearly/yr, … [4]

Pandas has a "date offset" "frequency string" microsyntax that supports business days, quarters, and years; e.g. BQuarterEnd, BQuarterBegin [5]

IDK how usable by other tools these date string parsers are.

W/ just a text editor, having `todo.txt`, `daily.todo.txt`, and `weekly.todo.txt` (and `cleanhome.todo.txt` and `hygiene.todo.txt` with "## heading" tasks that get lost @where +sorting) works okay.

I have physical 43 folders, too: A 12 month and a 31 day expanding file. [6]

[1] http://todotxt.org/

[2] https://taskwarrior.org/docs/using_dates.html

[3] https://taskwarrior.org/docs/named_dates.html

[4] https://taskwarrior.org/docs/durations.html

[5] https://pandas.pydata.org/pandas-docs/stable/user_guide/time...

[6] http://www.43folders.com/

[-]

Ask HN: Any suggestion on how to test CLI applications?

Hello HN!

I've been looking at alternatives on how to test command line applications, specifically, for example, exit codes, output messages and whatnot. I've seen "bats" https://github.com/sstephenson/bats and Bazel for testing but I'm curious as what other tools people use in a day to day basis. UI testing is nice with tools like Cypress.io and maybe there's something out there that isn't as popular but it's useful.

Thoughts?

pytest-docker-pexpect: https://github.com/nvbn/pytest-docker-pexpect

Pexpect: https://pexpect.readthedocs.io/en/stable/

pytest with subprocess.popen (or Sarge) may be sufficient for checking return codes and checking stdout and stderr output streams. Pytest has tmp_path and tmpdir fixtures that provide less test isolation than Docker containers: http://doc.pytest.org/en/latest/tmpdir.html

sarge.Capture.expect() takes a regex and returns None if there's no match: https://sarge.readthedocs.io/en/latest/tutorial.html#looking...

The Golden Butterfly and the All Weather Portfolio

The Golden Butterfly (is a modified All Weather Portfolio)

> Stocks: 20% Domestic Large Cap Fund (Vanguard’s VTI or Goldman Sach’s JUST), 20% Domestic Small Cap Value (Vanguard’s VBR)

> Bonds: 20% Long Term (Vanguard’s BLV), 20% Short Term (Vanguard’s BSV)

> Real Assets: 20% Gold (SPDR’s GLD)

The All Weather Portfolio:

> Stocks: 30% Domestic Total Stock Market (VG total stock)

> Bonds: 40% Long Term, 15% Intermediate-Term

> Real Assets: 7.5% Commodities, 7.5% Gold

What about investing in sustainable, innovative startups and small businesses (and crowdfunding campaigns)? What about direct capital investment? What about the American dream?

(Small businesses are a significant source of growth in our economy today and for the future)

[-]

Canada's Decision To Make Public More Clinical Trial Data Puts Pressure On FDA

We really could get more out of this data through international collaboration and through linked data (e.g. URIs for columns). See: "Open, and Linked, FDA data" https://github.com/FDA/openfda/issues/5#issuecomment-5392966... and "ENH: Adverse Event Count / 'Use' Count Heatmap" https://github.com/FDA/openfda/issues/49

With sales/usage counts, we'd have a denominator with which we could calculate relative hazard.

[-]

Python Alternative to Docker

Shiv does not solve for what containers and Docker/Podman/Buildah/Containerd solve for: re-launching processes at boot and failure, launching processes in chroots or cgroups (with least privileges), limiting access to network ports, limiting access to the host filesystem, building chroots / images, [...]

You can run build tools like shiv with a RUN instruction in a Dockerfile and get some caching.

You can build a zipapp with shiv (in a build container) and run the zipapp in a container.

Should the zipapp contain the test suite(s) and test_requires so that the tests can be run in an environment most similar to production?

It's much easier to develop with code on the filesystem (instead of in a zipapp).

It's definitely faster to read the whole zipapp into RAM than to stat and read each imported module from the filesystem once at startup.

There may be a better post title than the current "Python Alternative to Docker"? Shiv is a packaging utility for building Python zipapps. Shiv is not an alternative to process isolation with containers (or VMs)

[-]

$6B United Nations Agency Launches Bitcoin, Ethereum Crypto Fund

"UNICEF launches Cryptocurrency Fund: UN Children’s agency becomes first UN Organization to hold and make transactions in cryptocurrency" https://www.unicef.org/press-releases/unicef-launches-crypto...

From https://www.unicefusa.org/ :

> UNICEF USA helps save and protect the world's most vulnerable children. UNICEF USA is rated one of the best charities to donate to: 89% of every dollar spent goes directly to help children.

[-]

Supreme Court allows blind people to sue retailers if websites aren't accessible

"a11y": Accessibility

https://a11yproject.com/ has patterns, a checklist for checking web accessibility, resources, and events.

awesome-a11y has a list of a number of great resources for developing accessible applications: https://github.com/brunopulis/awesome-a11y

In terms of W3C specifications [1], you've got: WAI-ARIA (Web Accessibility Initiative: Accessibile Rich Internet Applications) [2], and WCAG: Web Content Accessibility Guidelines [3]. The new W3C Payment Request API [4] makes it easy for browsers to offer a standard (and probably(?) already accessible) interface for the payment data entry screen, at least.

There are a number of automated accessibility testing platforms. "[W3C WAI] Web Accessibility Evaluation Tools List" [5] lists quite a few. Can someone recommend a good accessibility testing tools? Is Google Lighthouse (now included with Chrome Devtools and as a standalone script) a good tool for accessibility reviews?

[1] https://github.com/brunopulis/awesome-a11y/blob/master/topic...

[2] https://www.w3.org/TR/using-aria/

[3] https://www.w3.org/WAI/standards-guidelines/wcag/

[4] https://www.w3.org/TR/payment-request/

[5] https://www.w3.org/WAI/ER/tools/

[-]

Streamlit: Turn a Python script into an interactive data analysis tool

Cool!

requests_cache caches HTML requests into one SQLite database. [1] pandas-datareader can cache external data requests with requests-cache. [2]

dask.cache can do opportunistic caching (of 2GB of data). [3]

How does streamlit compare to jupyter voila dashboards (with widgets and callbacks)? They just launched a new separate github org for the project. [4] There's a gallery of voila dashboard examples. [5]

> Voila serves live Jupyter notebooks including Jupyter interactive widgets.

> Unlike the usual HTML-converted notebooks, each user connecting to the Voila tornado application gets a dedicated Jupyter kernel which can execute the callbacks to changes in Jupyter interactive widgets.

> - By default, voila disallows execute requests from the front-end, preventing execution of arbitrary code.

[1] https://github.com/reclosedev/requests-cache

[2] https://pandas-datareader.readthedocs.io/en/latest/cache.htm...

[3] https://docs.dask.org/en/latest/caching.html

[4] https://github.com/voila-dashboards/voila

[5] https://blog.jupyter.org/a-gallery-of-voil%C3%A0-examples-a2...

Acess control and resource exhaustion are challenges with building any {Flask, framework_x,} app [from Jupyter notebooks]. First it's "HTTP Digest authentication should be enough for now"; then it's "let's use SSO and LDAP" (and review every release); then it's "why is it so sloww?". JupyterHub has authentication backends, spawners, and per-user-container/vm resource limits.

> Each user on your JupyterHub gets a slice of memory and CPU to use. There are two ways to specify how much users get to use: resource guarantees and resource limits. [6]

[6] https://zero-to-jupyterhub.readthedocs.io/en/latest/user-res...

Some notes re: voila and JupyterHub:

> The reason for having a single instance running voila only is to allow non JupyterHub users to have access to the dashboards. So without going through the Hub auth flow.

> What are the requirements in your case? Voila can be installed in the single user Docker image, so that each user can also use it on their own server (as a server extension for example). [7]

[7] https://github.com/voila-dashboards/voila/issues/112

[-]

Scott’s Supreme Quantum Supremacy FAQ

Who even asked these questions?

I question this. All of this.

[+]

I believe Feynman originally asked the QC question many many years ago. What an exciting milestone and a great FAQ.

"Always naysaying! Everything I create!"

[-]

Ask HN: How do you handle/maintain local Python environments?

I'm having some trouble figuring out how to handle my local Python. I'm not asking about 2 vs 3 - that ship has sailed - I'm confused on which binary to be using. From the way I see it, there's at least 4 different Pythons I could be using:

1 - Python shipped with OS X/Ubuntu

2 - brew/apt install python

3 - Anaconda

4 - Getting Python from https://www.python.org/downloads/

And that's before getting into how you get numpy et al installed. What's the general consensus on which to use? It seems like the OS X default is compiled with Clang while brew's version is with GCC. I've been working through this book [1] and found this thread [2]. I really want to make sure I'm using fast/optimized linear algebra libraries, is there an easy way to make sure? I use Python for learning data science/bioinformatics, learning MicroPython for embedded, and general automation stuff - is it possible to have one environment that performs well for all of these?

[1] https://www.amazon.com/Python-Data-Analysis-Wrangling-IPython/dp/1449319793

[2] https://www.reddit.com/r/Python/comments/46r8u0/numpylinalgsolve_is_6x_faster_on_my_mac_than_on/

[+]

I also prefer conda for the same reasons.

Precompiled MKL is really nice. Conda and conda-forge now build for aarch64. There are very few wheels for aarch64 on PyPI. Conda can install things like Qt (IPython-qt, spyder,) and NodeJS (JupyterLab extensions).

If I want to switch python versions for a given condaenv (instead of just creating a new condaenv for a different CPython/PyPy version), I can just run e.g. `conda install -y python=3.7` and it'll reinstall everything in the depgraph that depended on the previous python version.

I always just install miniconda instead of the whole anaconda distribution. I always create condaenvs (and avoid installing anything in the root condaenv) so that I can `conda-env export -f environment.yml` and clean that up.

BinderHub ( https://mybinder.org/ ) creates docker containers from {git repos, Zenodo, FigShare,} and launches them in free cloud instances also running JupyterLab by building containers with repo2docker (with REES (Reproducible Execution Environment Specification)). This means that all I have to do is add an environment.yml to my git repo in order to get Binder support so that people can just click on the badge in the README to launch JupyterLab with all of the dependencies installed.

REES supports a number of dependency specifications: requirements.txt, Pipfile.lock, environment.yml, aptSources, postBuild. With an environment.yml, I can install the necessary CPython/PyPy version and everything else.

...

In my dotfiles, I have a setup_miniconda.sh script that installs miniconda into per-CPython-version CONDA_ROOT and then creates a CONDA_ENVS_PATH for the condaenvs. It may be overkill because I could just specify a different python version for all of the conda envs in one CONDA_ENVS_PATH, but it keeps things relatively organized and easily diffable: CONDA_ROOT="~/-wrk/-conda37" CONDA_ENVS_PATH="~/-wrk/-ce37"

I run `_setup_conda 37; workon_conda|wec dotfiles` to work on the ~/-wrk/-ce37/dotfiles condaenv and set _WRD=~/-wrk/-ce37/dotfiles/src/dotfiles.

Similarly, for virtualenvwrapper virtualenvs, I run `WORKON_HOME=~/-wrk/-ve37 workon|we dotfiles` to set all of the venv cdaliases; i.e. then _WRD="~/-wrk/-ve37/dotfiles/src/dotfiles" and I can just type `cdwrd|cdw` to cd to the working directory. (Some of the other cdaliases are: {cdwrk, cdve|cdce, cdvirtualenv|cdv, cdsrc|cds}. So far, I have implemented cdalias support for bash, IPython, and vim)

One nice thing about defining _WRD is I can run `makew <tab>` and `gitw` to `cd $_WRD; make <tab>` and `git -C $_WRD` without having to change directory and then `cd -` to return to where I was.

So, for development, I use a combination of virtualenvwrapper, pipsi, conda, and some shell scripts in my dotfiles that I should get around to releasing and maintaining someday. https://westurner.github.io/dotfiles/venv

For publishing projects, I like environment.yml because of the REES support.

[-]

Is the era of the $100 graphing calculator coming to an end?

For $100, you can buy a Pinebook with an 11" or 14" screen, a multitouch trackpad, gigabytes of storage, WiFi, a keyboard without a numpad, and an ARM processor.

On this machine, you can create reproducible analyses with JupyterLab; do arithmetic with Python; work with multidimensional arrays with NumPy, SciPy, Pandas, xarray, Dask; do machine learning with Statsmodels, Scikit-learn, Dask-ML, TPOT; create books of these notebooks (containing code and notes (in Markdown, which easily transformed to HTML) and LaTeX equations) with jupyter-book, nbsphinx, git + BinderHub; store the revision history of your discoveries; publish what you've discovered and learned to public or private git repositories; and complete graded exercises with nbgrader.

But the task is to prepare for a world of mental arithmetic, no validation, no tests, no reference materials, and no search engines; and CAS (Computer Algebra Systems) systems like SymPy and Sage are not allowed.

On this machine, you can run write code, write papers, build spreadsheets and/or Jupyter notebooks, run physical simulations, explore the stars, and play games and watch videos. Videos like: Khan Academy videos and exercises that you can watch and do, with validation, until you've achieved mastery and move on to the next task on your todo.txt list.

But the task is to preserve your creativity and natural curiosity despite the compulsory education system's demands for quality control and allocative efficiency; in an environment where drama and popularity are the solutions to relatedness and acceptance needs.

I have three of these $100 calculators in my toolbox. It's been so long since I've powered them on that I'm concerned that the rechargeable AAA batteries are leaking battery acid.

For $100, you can buy an ARM notebook and install conda and conda-forge packages and build sweet visualizations to collaborate with colleagues on (with Seaborn (matplotlib), HoloViews, Altair, Plotly)

"You must buy a $100 calculator that only runs BASIC and ASM, and only use it for arithmetic so that we can measure you."

Hand tools are fun, but please don't waste any more of my compulsory time.

[+]
[-]

Reinventing Home Directories

[+]
[+]
[+]
[+]
[+]

Why do you think that is? Why have production grade Linux distributions all chosen to adopt systemd?

With SysV init, how do you securely launch processes in cgroups, such that they'll consistently restart when the process happens to terminate, with stdout and stderr logged with consistent timestamps, with process dependency models that allow for faster boots due to parallelization?

(edit)

Journalctl is far better than `tail -f /var/log/starstar` and parsing all of those timestamps and inconsistently escaped logfile formats. There's no good way to modify everything in /etc/init.d in order to log to syslog-ng or rsyslog. Systemd and journalctl solve for that; for unified logging.

IMO, there's no question that systemd is the better way and I have zero nostalgia for spawning everything from a probably a shell specified in an /etc/init.d shebang without process restarting (and logging thereof), cgroups, and consistent logging.

[+]

When journald logfile corruption occurs, it's detected and it starts writing a new logfile.

When flatfile logfile corruption occurs, it's not detected and there are multiple logfile formats to contend with. And multiple haphazard logrotate configs.

Here's how to use a separate process to ship journald logs - from one file handle - to a remote logging service: https://unix.stackexchange.com/questions/394822/how-should-i...

While there is a systemd-journal-remote, it's not necessary for journald to try and replicate what's already solved and tested in rsyslog and syslog-ng.

It's quite a bit more work to add every new service to the syslog-ng or rsyslog configuration than to just ship one journald log.

Furthermore, service start/stop events are already in the same stream (with the same timestamp format) with the services' stdout and stderr.

Why hasn't anyone written fsck for corrupted journald recovery?

...

I have not needed to makedev and chown and chatted and chcon anything in very many years. When you accidentally newbishly delete something from a static /dev and rebooting doesn't work and you have no idea what the major minor is or was, it sucks bad.

When you're trying to boot a system on a different machine but it doesn't work because the NIC is in a different bus, it's really annoying to have to symlink /dev or modify /etc. With udevd, all you need to do is define a rule to map the busid device name to e.g. eth0. I can remember encountering the devfs race condition resulting in eth0 and eth1 being mapped to different devices on different boots; which was dangerous because firewall rules are applied to device names.

Udev has been in the kernel since 2.6.

"What problems does udev actually solve?" https://superuser.com/questions/686774/what-problems-does-ud...

With integrated udev and systemd, I have no reason to run a separate hotplugd with a different config format (again with no cgroup support) and a different logstream.

Perhaps ironically, here's a link to the presentation PDF that was posted yesterday: https://news.ycombinator.com/item?id=21036020

And my comments there:

> What a good idea.

> Here's the hyperlinkified link to the {systemd-homed.service, systemd-userdbd.service, homectl, userdbctl} sources from the PDF: https://github.com/poettering/systemd/tree/homed

> Hadn't heard of varlink: https://varlink.org/

> Is there a FIPS-like subset of the most-widely-available LUKS configs? Otherwise home directories won't work on systems that have a limited set of LUKS modules.

[-]

Serverless: slower and more expensive

It'd be interesting to see how much this same workload would cost with e.g. OpenFaaS on k8s with autoscaling to zero; but there also you'd need to include maintenance costs like OS and FaaS stack upgrades. https://docs.openfaas.com/architecture/autoscaling/

[-]

Entropy can be used to understand systems

Maximum entropy: https://en.wikipedia.org/wiki/Maximum_entropy

Here's a quote of a tweet about a (my own): comment on a schema:BlogPost: https://twitter.com/westurner/status/1048125281146421249:

> “When Bayes, Ockham, and Shannon come together to define machine learning” https://towardsdatascience.com/when-bayes-ockham-and-shannon...

> Comment: "How does this relate to the Principle of Maximum Entropy? How does Minimum Description Length relate to Kolmogorov Complexity?"

[+]
[-]

New Query Language for Graph Databases to Become International Standard

Graph query languages are nice and all, but what about Linked Data here? Queries of schemaless graphs miss lots of data because without a schema this graph calls it "color" and that graph calls it "colour" and that graph calls it "色" or "カラー". (Of course this is also an issue even when there is a defined schema; but it's hardly possible to just happen to have comprehensible inter or even intra-organizational cohesion without e.g. RDFS and/or OWL and/or SHACL for describing (and changing) the shape of the data)

So, the task is then to compile schema-aware SPARQL to GQL or GraphQL or SQL or interminable recursive SQL queries or whatever it is.

For GraphQL, there's GraphQL-LD (which somewhat unfortunately contains a hashtag-indeterminate dash). I cite this in full here because it's very relevant to the GQL task at hand:

"GraphQL-LD: Linked Data Querying with GraphQL" (2018) https://comunica.github.io/Article-ISWC2018-Demo-GraphQlLD/

> GraphQL is a query language that has proven to be a popular among developers. In 2015, the GraphQL framework [3] was introduced by Facebook as an alternative way of querying data through interfaces. Since then, GraphQL has been gaining increasing attention among developers, partly due to its simplicity in usage, and its large collection of supporting tools. One major disadvantage of GraphQL compared to SPARQL is the fact that it has no notion of semantics, i.e., it requires an interface-specific schema. This therefore makes it difficult to combine GraphQL data that originates from different sources. This is then further complicated by the fact that GraphQL has no notion of global identifiers, which is possible in RDF through the use of URIs. Furthermore, GraphQL is however not as expressive as SPARQL, as GraphQL queries represent trees [4], and not full graphs as in SPARQL.

> In this work, we introduce GraphQL-LD, an approach for extending GraphQL queries with a JSON-LD context [5], so that they can be used to evaluate queries over RDF data. This results in a query language that is less expressive than SPARQL, but can still achieve many of the typical data retrieval tasks in applications. Our approach consists of an algorithm that translates GraphQL-LD queries to SPARQL algebra [6]. This allows such queries to be used as an alternative input to SPARQL engines, and thereby opens up the world of RDF data to the large amount of people that already know GraphQL. Furthermore, results can be translated into the GraphQL-prescribed shapes. The only additional requirement is their queries would now also need a JSON-LD context, which could be provided by external domain experts.

> In related work, HyperGraphQL [7] was introduced as a way to expose access to RDF sources through GraphQL queries and emit results as JSON-LD. The difference with our approach is that HyperGraphQL requires a service to be set up that acts as a intermediary between the GraphQL client and the RDF sources. Instead, our approach enables agents to directly query RDF sources by translating GraphQL queries client-side.

All of these RDFS vocabularies and OWL ontologies provide structure that minimizes the costs of merging and/or querying multiple datasets: https://lov.linkeddata.es/dataset/lov/

All of these schema.org/Dataset s in the "Linked Open Data Cloud" are easier to query than a schemaless graph: https://lod-cloud.net/ . Though one can query schemaless graphs with SPARQL, as well.

For reference, RDFLib has a bunch of RDF graph implementations over various key/value and SQL store backends. RDFLib-sqlachemy does query parametrization correctly in order to minimize the risk of query injection. FOR THE RECORD, SQL Injection is the CWE Top 25 #1 most prevalent security weakness; which is something that any new spec and implementation should really consider before launching anything other than an e.g. overly-verbose JSON-based query language that people end up bolting a micro-DSL onto. https://github.com/RDFLib/rdflib-sqlalchemy

Most practically, I frequently want to read a graph of objects into RAM; update, extend, and interlink; and then transactionally save the delta back to the store. This requires a few things: (1) an efficient binary serialization protocol like Apache Arrow (SIMD), Parquet, or any of the BSON binary JSONs; (2) a transactional local store that can be manually synchronized with the remote store until it's consistent.

SPARQL Update was somewhat of an out-of-scope afterthought. Here's SPARQL 1.1 Update: https://www.w3.org/TR/sparql11-update/

Here's SOLID, which could be implemented with SPARQL on GQL, too; though all the re-serialization really shouldn't be necessary for EAV triples with a named graph URI identifier: https://solidproject.org/

5 star data: PDF -> XLS -> CSV -> RDF (GQL, AFAIU (but with no URIs(!?))) -> LOD https://5stardata.info/en/

[+]

> Linked Data tends to live in a semantic web world that has a lot of open world assumptions. While there are a few systems like this out there, there aren't many. More practically focused systems collapse this worldview down into a much simpler model, and property graphs suit just fine.

Data integration is cost prohibitive. In n years time, the task is "Let's move all of these data silos into a data lake housed in our singular data warehouse; and then synchronize and also copy data around to efficiently query it in one form or another"

Linked data enables data integration from day one: enables the linking of tragically-silo'd records within disparate databases

There are very very many systems that share linked data. Some only label some of the properties with URIs in templates. Some enable federated online querying.

When you develop a schema for only one application implementation, you're tragically limiting the future value of the data.

> There's nothing wrong with enabling linked data use cases, but you don't need RDF+SPARQL+OWL and the like to do that.

Can you name a property graph use case that cannot be solved with RDFS and SPARQL?

> The "semantic web stack" I think has been shown by time and implementation experience to be an elegant set of standards and solutions for problems that very few real world systems want to tackle.

TBH, I think the problem is that people don't understand the value in linking our data silos through URIs; and so they don't take the time to learn RDFS or JSON-LD (which is pretty simple and useful for very important things like SEO: search engine result cards come from linked data embedded in HTML attributes (RDFa, Microdata) or JSON-LD)

The action buttons to 'RSVP', 'Track Package', anf 'View Issue' on Gmail emails are schema.org JSON-LD.

Applications can use linked data in any part of the stack: the database, the messages on the message queue, in the UI.

You might take a look at all of the use cases that SOLID solves for and realize how much unnecessary re-work has gone into indexing structs and forms validation. These are all the same app with UIs for interlinked subclasses of https://schema.org/Thing with unique inferred properties and aggregations thereof.

> In the intervening 2 full generations of tech development that have happened since a lot of those standards were born, some of the underlying stuff too (most particularly XML and XML-NS) went from indispensable to just plain irritating.

Without XSD, for example, we have no portable way to share complex fractions.

There's a compact representation of JSON-LD that minimizes record schema overhead (which gzip or lzma generally handle anyway)

https://lod-cloud.net is not a trivial or insignificant amount of linked data: there's real value in structuring property graphs with standard semantics.

Are our brains URI-labeled graphs? Nope, and we spend a ton of time talking to share data. Eventually, it's "well let's just get a spreadsheet and define some columns" for these property graph objects. And then, the other teams' spreadsheets have very similar columns with different labels and no portable datatypes (instead of URIs)

[+]
[+]
[+]

What was the vision?

The RDFJS "Comparison of RDFJS libraries" wiki page lists a number of implementations; though none for React or AngularJS yet, unfortunately. https://www.w3.org/community/rdfjs/wiki/Comparison_of_RDFJS_...

There's extra work to build general purpose frameworks for Linked Data. It may have been hard for any firm with limited resources to justify doing it the harder way (for collective returns)

Dokieli (SOLID (LDP,), WebID, W3C Web Annotations,) is a pretty cool - if deceptively simple-looking - showcase of what's possible with Linked Data; it just needs some CSS and a revenue model to pay for moderation. https://dokie.li/

> property graphs are demonstrably easier to work with for most use cases.

How do you see property graphs as distinct from RDF?

People build terrible apps without schema or validation and leave others to clean that up.

[+]

I added an answer in context to the comments on the answer you've linked but didn't add a link from the comments to the answer. Here's that answer:

> (in reply to the comments on this answer: https://stackoverflow.com/a/30167732 )

> When an owl:inverseOf production rule is defined, the inverse property triple is inferred by the reasoner either when adding or updating the store, or when selecting from the store. This is a "materialized relation"

> Schema.org - an RDFS vocabulary - defines, for example, https://schema.org/isPartOf as the inverse property of hasPart. If both are specified, it's not necessary to run another graph pattern query to traverse a directed relation in the other direction. (:book1 schema:hasPart ?o), (?o schema:isPartOf :book1), (?s schema:hasPart :chapter2)

> It's certainly possible to use RDFS and OWL to describe schema for and within neo4j property graphs; but there's no reasoner to e.g. infer inverse properties or do schema validation.

> Is there any RDF graph that neo4j cannot store? RDF has datatypes and languages for objects: you'd need to reify properties where datatypes and/or languages are specified (and you'd be re-implementing well-defined semantics)

> Can every neo4j graph be represented with RDF? Yes.

> RDF is a representation for graphs for which there are very many store implementations that are optimized for various use cases like insert and query performance.

> Comparing neo4j to a particular triplestore (with reasoning support) might be a more useful comparison given that all neo4j graphs can be expressed as RDF.

And then, some time later, I realize that I want/need to: (3) apply production rules to do inference at INSERT/UPDATE/DELETE time or SELECT time (and indicate which properties were inferred (x is a :Shape and a :Square, so x is also a :Rectangle; x is a :Rectangle and :width and :height are defined, so x has an :area)); (4) run triggers (that execute code written in a different language) when data is inserted, updated, modified, or linked to; (5) asynchronously yield streaming results to message queue subscribers who were disconnected when the cached pages were updated

[-]

A Python Interpreter Written in Python

What an excellent 500 lines introduction to the byterun bytecode interpreter / virtual machine: https://github.com/nedbat/byterun

Also, proceeds from optional purchases of the AOSA books go to Amnesty International. https://aosabook.org/

[-]

Reinventing Home Directories – systemd-homed [pdf]

What a good idea.

Here's the hyperlinkified link to the {systemd-homed.service, systemd-userdbd.service, homectl, userdbctl} sources from the PDF: https://github.com/poettering/systemd/tree/homed

Hadn't heard of varlink: https://varlink.org/

Is there a FIPS-like subset of the most-widely-available LUKS configs? Otherwise home directories won't work on systems that have a limited set of LUKS modules.

[-]

Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM

There's also RustPython, a Rust implementation of CPython 3.5+: https://news.ycombinator.com/item?id=20686580

> https://github.com/RustPython/RustPython

[+]
[+]
[+]
[+]
[+]
[+]
[+]
[+]
[+]
[+]
[+]
[-]

Craftsmanship–The Alternative to the 4 Hour Work Week

> To be successful over the course of a career requires the application and accumulation of expertise. This assumes that for any given undertaking you either provide expertise or you are just a bystander. It’s the experts that are the drivers — an expertise that is gained from a curiosity, and a mindset of treating one’s craft very seriously.

[-]

Solar and Wind Power So Cheap They’re Outgrowing Subsidies

[+]
[+]
[+]
[+]
[+]
[+]
[+]
[+]
[+]
[+]

No, you're splitting hairs.

There are direct and indirect subsidies. Indirect subsidies include externalities: external costs paid by everyone else (that the government should be incentivizing reductions in by requiring the folks causing them to pay)

Semantic digressions aside, they're earning while everyone else pays costs resultant from their operations (and from our apparent inability to allocate with e.g. long term security, health, and prosperity as primary objectives for the public sphere)

[+]
[+]
[+]
[+]
[+]

"subsidies" includes both direct and indirect subsidies.

We can measure direct subsidies by measuring real and effective tax rates.

We can measure indirect subsidies like healthcare costs paid by Medicare with subjective valuations of human life and rough estimates of the value of a person's health and contribution to growth in GDP, and future economic security.

But who has the time for this when we're busy paying to help folks who require disaster relief services from the government and NGOs (neither of which are preventing further escalations in costs)

[+]
[-]

Show HN: Python Tests That Write Themselves

[+]
[+]

pytype (Google) [1], PyAnnotate (Dropbox) [2], and MonkeyType (Instagram) [3] all do dynamic / runtime PEP-484 type annotation type inference [4]

[1] https://github.com/google/pytype

[2] https://github.com/dropbox/pyannotate

[3] https://github.com/Instagram/MonkeyType

[4] https://news.ycombinator.com/item?id=19454411

[-]

Most Americans see catastrophic weather events worsening

The stratifications on this are troubling.

> But there are wide differences in assessments by partisanship. Nine in 10 Democrats think weather disasters are more extreme, compared with about half of Republicans.

It's not a partisan issue: we all pay these costs.

> Majorities of adults across demographic groups think weather disasters are getting more severe, according to the poll. College-educated Americans are slightly more likely than those without a degree to say so, 79 percent versus 69 percent.

Weather disasters are getting more severe. It is objectively, quantitatively true that weather disasters are getting more frequent and more severe.

[+]

> Source? What definitions are being used for severity? How is the sample of events selected? Is there a statistically-significant effect or might it be random variation?

These are great questions that any good skeptic / data scientist should always be asking. Here are some summary opinions based upon meta analyses with varyingly stringent inclusion criteria.

( I had hoped that the other top-level post I posted here would develop into a discussion, but these excerpts seem to have bubbled up. https://news.ycombinator.com/item?id=20919368 )

"Scientific consensus on climate change" lists concurring, non-commital, and opposing groups of persons with and without conflicting interests: https://en.wikipedia.org/wiki/Scientific_consensus_on_climat...

USGCRP, "2017: Climate Science Special Report: Fourth National Climate Assessment, Volume I" [Wuebbles, D.J., D.W. Fahey, K.A. Hibbard, D.J. Dokken, B.C. Stewart, and T.K. Maycock (eds.)]. U.S. Global Change Research Program, Washington, DC, USA, 470 pp, doi: 10.7930/J0J964J6.

"Chapter 8: Droughts, Floods, and Wildfire" https://science2017.globalchange.gov/chapter/8/

"Chapter 9: Extreme Storms" https://science2017.globalchange.gov/chapter/9/

"Appendix A: Observational Datasets Used in Climate Studies" https://science2017.globalchange.gov/chapter/appendix-a/

The key findings in this report do list supporting evidence and degrees of confidence in predictions about the frequency and severity of severe weather events.

I'll now proceed to support the challenged claim that disaster severity and frequency are increasing by citing disaster relief cost charts which do not directly support the claim. Unlike your typical televised debate or congressional session, I have: visual aids, a computer, linked to the sources I've referenced. Finding the datasets ( https://schema.org/Dataset ) for these charts may be something that someone has time for while the costs to taxpayers and insurance holders are certainly increasing for a number of reasons.

"Taxpayer spending on U.S. disaster fund explodes amid climate change, population trends" (2019) has a nice chart displaying "Disaster-relief appropriations, 10-year rolling median" https://www.washingtonpost.com/us-policy/2019/04/22/taxpayer...

"2018's Billion Dollar Disasters in Context" includes a chart from NOAA: "Billion-Dollar Disaster Event Types by Year (CPI-Adjusted)" with the title embedded in the image text - which I searched for - and eventually found the source of: [1] https://www.climate.gov/news-features/blogs/beyond-data/2018...

[1] "Billion-Dollar Weather and Climate Disasters: Time Series" (1980-2019) https://www.ncdc.noaa.gov/billions/time-series

[+]
[+]

The article seems to have focused on perceptions of persons who aren't concerned with taking an evidence-based look (at various types of storms: floods, cyclones (i.e. hurricanes), severe thunderstorms, windstorms. Regardless, costs are increasing. I've listed a few sources here: https://news.ycombinator.com/item?id=20925127

"2017: Climate Science Special Report: Fourth National Climate Assessment, Volume I" > "Chapter 9: Extreme Storms" lists a number of relevant Key Findings with supporting evidence (citations) and degrees of confidence: https://science2017.globalchange.gov/chapter/9/

[+]

"Billion-Dollar Weather and Climate Disasters: Time Series" (1980-2019) https://www.ncdc.noaa.gov/billions/time-series

[+]

> My guess is you and I see very different things despite consuming the very same article. I lean conservative/libertarian (generally speaking),

HN specifically avoids politics. In context to the in-scope article, when you say "conservative/libertarian" do you mean: fiscally conservative (haven't seen a deficit hawk in decades other than "Read my lips. No new taxes" followed by responsibly raising taxes), socially libertarian (Liberty as a fundamental right; if you're not violating the rights of others the government is not obligated or even granted the right to intervene at all), or conservative as in imposing your particular traditional standard of moral values which you believe are particular to a particular side of the aisle?

Or, do you mean that you're libertarian in regards to the need and the right to regulate business and industry in the interest of consumers ("laissez faire")? I'm certainly not the only person to observe that lack of regulation results in smog-filled cities due to un-costed 'externalities' in a blind pursuit of optimization for short-term profit.

At issue here, I think, is whether we think we can avert future escalations of costs by banding together to address climate change now; and how best to achieve the Paris Agreement targets that we set for ourselves (despite partisan denial, delusion, and indifference to increasing YoY costs [1]) https://en.wikipedia.org/wiki/Paris_Agreement

I'm personally and financially far more concerned about the long-term costs of climate change than a limited number of special interests who can very easily diversify and/or divest to take advantage of the exact same opportunities.

> and I am deeply distrustful of government (for extremely good reasons I believe), so I know for a fact that my interpretation of the article is going to be heavily distorted by that. Any logical inconsistency, ambiguousness, disingenuousness, technical dishonesty, or anything else along those lines is going to get red flagged in my mind, whereas others will read it in a much more forgiving fashion. And in an article on a different political hot topic, we will switch our behaviors.

While governments (and militaries (TODO)) do contribute substantially to emissions and resultant climate change, I think it unnecessary to qualify that unregulated decisions by industry should be the primary focus here. Industry has done far more to cause climate change than governments (which can more efficiently provide certain services useful to all citizens)

> In such threads, I think it would be extremely interesting for people with opposing views to post excerpts of the parts that "catch your attention", with an explanation of why. This is kind of what happens anyway, but I'm thinking with a completely different motive: rather than quoting excerpts with commentary to argue your ~political side of the issue with the goal of "winning the argument", take an unemotional, more abstract view of your personal cognitive processing of the article,

These people aren't doing jack about the problem because they haven't reviewed this chart: "Billion-Dollar Weather and Climate Disasters: Time Series" (1980-2019) https://www.ncdc.noaa.gov/billions/time-series

Maybe they want insurance payouts, which result in higher premiums. Maybe the people who built in those locations should be paying the costs.

> and post commentary on ~why/how you believe you feel you consider that important on a psychological level. Psychological self-analysis is famously difficult, but even with moderate success I suspect some very interesting things would rise to the surface.*

They don't even care because they refuse to accept that it's a problem.

The article was ineffectual at addressing the very real problem.

From https://news.ycombinator.com/item?id=20925127 :

> ( I had hoped that the other top-level post I posted here would develop into a discussion, but these excerpts seem to have bubbled up. https://news.ycombinator.com/item?id=20919368 )

In this observational study of perceptions, college education was less predictive than party affiliation.

Maybe reframing this as a short-term money problem [1] would result in compassion for people who are suffering billions of dollars of loss every year.

[+]
[+]

> "2017: Climate Science Special Report: Fourth National Climate Assessment, Volume I" > "Chapter 9: Extreme Storms" lists a number of relevant Key Findings with supporting evidence (citations) and degrees of confidence: https://science2017.globalchange.gov/chapter/9/

How about a link to a chart indicating frequency and severity of severe weather events?

The Paris Agreement is predicated upon the link between human actions, climate change, and severe weather events. 195 countries have signed the Paris Agreement with consensus that what we're doing is causing climate change.

Here are some climate-relevant poll questions:

Do you think the costs of disaster relief will continue to increase due to frequency and severity of severe weather events?

Does it make sense to spend more on avoiding further climate change now rather than even more on disaster relief later?

How can you help climate refugees? Do you donate to DoD and National Guards? Do you donate to NGOs? How can we get better at handling more frequent and more severe disasters?

[-]

Emergent Tool Use from Multi-Agent Interaction

gdb | 2019-09-17 12:00:54 | 332 | # | ^

I, for one, really appreciate the raytracing in these visualizations. I wish for more box surfing examples.

[-]

Inkscape 1.0 Beta 1

`Ctrl + 4` to center view on page!

Pressure sensitive pencil for the PowerStroke Live Path Effect (LPE) "if a pressure sensitive device is available"!

[-]

Where Dollar Bills Come From

The 1914 $10 Dollar Bill was printed on hemp paper. Today, they're worth like $49.99. IDK how steady that price is over time; relative to the prices of other CPI All goods.

[+]
[+]
[-]

Monetary Policy Is the Root Cause of the Millennials’ Struggle

Volatility works out for people who save (who park capital in liquid assets that aren't doing work in order to have wheat for the eventual famine). These guys. They save, short like heck when the market is falling, and swoop in to save the day. What a great time to be selling 0% loans.

Personal Savings Rate (PSR) stratified by greatest generation and not greatest generation is also relevant. Are relatively fixed living expenses higher now? Yes. Is my generation just blowing what they could invest into interest-bearing investments on unnecessary stuff from Amazon? Yes. And expensive meals and drinks.

How have corporate profits and wages changed?

In their day, you put you gosh-danged money aside. For later. So that you have money later.

And that is why you should buy my book, entitled: "Invest in things with long term returns: don't buy shtuff you don't f need, save for tomorrow; and other financial advice"

Which brings me to: the cost of college textbooks and a college education in terms of average hourly wages.

By the way, over the longer term, index funds are likely to outperform funds. Gold may be likely to outperform the stock market. And, over the recent term -- this is for all you suckers out there -- cryptocurrencies have outperformed all stock and commodities markets. How much total wealth is being created on an annual basis here?

Payday loans have something like 300% APY.

How does 2% inflation affect trade when other central banking cabals haven't chosen the same target? "Devaluation"! "Treachery"!

[-]

Non-root containers, Kubernetes CVE-2019-11245 and why you should care

> At the same time, all the current implementations of rootless containers rely on user namespaces at their core. Not to be confused with what is referred to as non-root containers in this article, rootless containers are containers that can be run and managed by unprivileged users on the host. While Docker and other runtimes require a daemon running as root, rootless containers can be run by any user without additional capabilities.

non-root / rootless

[-]

How do black holes destroy information and why is that a problem?

[+]

"Why Quantum Information is Never Destroyed" re: determinism and T-Symmetry ("time-reversal symmetry") by PBS SpaceTime https://youtu.be/HF-9Dy6iB_4

Classical information is 'collapsed' quantum information, so that would mean that classical information is never lost either.

There appear to be multiple solutions for Navier-Stokes; i.e. somewhat chaotic.

If white holes are on the other side of black holes, Hawking radiation would not account for the entirety of the collected energy/information. Is our visible universe within a white hole? Is everything that's ever been embedded in the sidewall of a black hole shredder?

Maybe even recordings of dinosaurs walking; or is that lemurs walking in reverse?

Do 1/n, 1/∞, and n/∞ approach a symbolic limit where scalars should not be discarded; with piecewise operators?

[-]

Banned C standard library functions in Git source code

FWIW, here's awesome-static-analysis > Programming Languages > C/C++: https://github.com/mre/awesome-static-analysis/blob/master/R...

These tools have lists of functions not to use. Most of them — at least the security-focused ones — likely also include: strcpy, strcat, strncpy, strncat, sprints, and vsprintf just like banned.h

[-]

Ask HN: What's the hardest thing to secure in a web-app?

"OWASP Top 10 Most Critical Web Application Security Risks" https://www.owasp.org/index.php/Category:OWASP_Top_Ten_Proje...

> A1:2017-Injection, A2:2017-Broken Authentication, A3:2017-Sensitive Data Exposure, A4:2017-XML External Entities (XXE), A5:2017-Broken Access Control, A6:2017-Security Misconfiguration, A7:2017-Cross-Site Scripting (XSS), A8:2017-Insecure Deserialization, A9:2017-Using Components with Known Vulnerabilities, A10:2017-Insufficient Logging&Monitoring

"OWASP Top 10 compared to SANS CWE 25" https://www.templarbit.com/blog/2018/02/08/owasp-top-10-vs-s...

[-]

Crystal growers who sparked a revolution in graphene electronics

> This seven-metre-tall machine can squeeze carbon into diamonds

OT but, is this a thing now? Diamonds can be entangled.

[+]

Does it take more energy than mining for diamonds?

> Quantum Entanglement Links 2 Diamonds: Usually a finicky phenomenon limited to tiny, ultracold objects, entanglement has now been achieved for macroscopic diamonds at room temperature (2011) https://www.scientificamerican.com/article/room-temperature-...

[+]
[-]

Things to Know About GNU Readline

I map <up> to history-search-backward in my .inputrc; so I can type 'sudo ' and press <up> to cycle through everything starting with sudo:

    #  <up>      -- history search backward (match current input)
    "\e[A": history-search-backward
    #  <down>    -- history search forward (match current input)
    "\e[B": history-search-forward
https://github.com/westurner/dotfiles/blob/develop/etc/.inpu...

[+]
[+]
[+]
[+]
[+]
[+]
[+]
[+]
[+]
[+]
[+]
[+]

Is this macro from the article dangerous because it doesn't quote the argument?

  Control-j: "\C-a$(\C-e)"
I can never remember how expansion and variable substitution work in shells.

[+]

Yeah, but this and this do different things:

  # prints a newline
  echo $(echo "-e a\nb")

  # prints "-e a\nb"
  echo "$(echo "-e a\nb")"

[-]

Show HN: Termpage – Build a webpage that behaves like a terminal

This looks useful.

FWIW, you can build a curses-style terminal GUI with Urwid (in Python) and use that through the web. AFAIU, it requires Apache; but it's built on Tornado (which is now built on Asyncio) so something more lightweight than Apache on a Pi should definitely be doable. Termpage with like a Go or Rust REST API may still be more lightweight, but more work.

[-]

Vimer - Avoid multiple instances of GVim with gvim –remote[-tab]-silent wrapper

I have a shell script I named 'e' (for edit) that does basically this. If VIRTUAL_ENV_NAME is set (by virtualenvwrapper), e opens a new tab in that gui vim remote if gvim or macvim are on PATH, or just in a console vim if not. https://github.com/westurner/dotfiles/blob/develop/scripts/e

'editwrd'/'ewrd'/'ew' does tab-completion relative to whatever $_WRD (working directory) is set to (e.g. by venv) and calls 'e' with that full path: https://github.com/westurner/dotfiles/blob/develop/scripts/_...

It's unfortunately not platform portable like vimer, though.

[-]

Electric Dump Truck Produces More Energy Than It Uses

What a cool use of gravitational potential energy. It would be interesting to learn how much more energy is produced by the regenerative breaking system on the downhill and whether they use the excess to load the truck?

[-]

Ask HN: Let's make an open source/free SaaS platform to tackle school forms

I have 4 kids. I am filling out all the start of school forms for each kid. I have to fill out these same forms each year. Are you doing the same thing? Let's make this year the last year we are manually filling out forms -- let's build a SaaS platform for school forms. Community built, open-sourced, free.

Brief sketch of the idea: survey monkey + docusign, but with a 100 pre-built templates for K-12 school situations. Medical emergency form. Carpool form. Field trip permission form. Backend gives schools an easy way to customize and track forms. Forms are emailed to parents and filled out online. Parent's information is saved so that any new form is pre-filled in with as much known info as possible.

Anyone feeling the same pain? Anyone want to join with me and do it?

Technically, a checkbox may qualify as a digital signature; however, identification / authentication and storage integrity are fairly challengeable (just as a written signature on a piece of paper with a date written on it is challengeable)

Given that notarization is not required for parental consent forms, I'm not sure what sort of server security expense is justified or feasible.

How much does processing all of the paper forms cost each school? Per-student?

In terms of storing digital record of authorization, a private set of per-student OpenBadges with each OpenBadge issued by the school would be easy enough. W3C Verified Claims (and Linked Data Signatures) are the latest standards for this sort of thing.

We could evaluate our current standards for chain of custody in regards to the level of trust we place in commercial e-signature platforms.

The school could send home a sheet with a QR code and a shorturl, but that would be more expensive than running hundreds of copies of the same sheet of paper.

The school could require a parent or guardian's email address for each student in the SIS Student Information System and email unique links to prefilled forms requesting authorization(s).

Just as with e-Voting, assuring that the person who checks a checkbox or tries to scribble their signature with a mouse or touchscreen is the authorized individual may be more difficult than verifying that a given written signature is that of the parent or guardian authorized to authorize.

AFAIU, Google Forms for School can include the logged-in user's username; but parents don't have school domain accounts with Google Apps for Education or Google Classroom.

How would the solution integrate with schools' existing SIS (Student Information Systems)? Upload a CSV of (student, {student info}, {guardian email (s)})? This is private information that deserves security, which costs money.

Which users can log-in for the school and/or district to check the state of the permission / authorization requests and PII personally-identifiable information.

While cryptographic signatures may be overkill as a substitute for permission slips, FWIW, a timestamp within a cryptographically-signed document only indicates what the local clock was set to at the time. Blockchains have relatively indisputable timestamps ("certainly no later than the time that the tx made it into a block"), but blockchains don't solve for proving the key-person relation at a given point in time.

And also, my parent or guardian said you can take me on field trips if you want. https://backpack.openbadges.org/

[-]

Ask HN: Is there a CRUD front end for databases (especially SQLite)?

I'm currently looking for a program (a simple executable) that "opens" an SQLite database and (via introspection of the schema) without any further configuration allows simple CRUD operations on the database.

Yes, there is DB Browser and a gazillion other database administration frontends, but it should really be limited to CRUD operations. No changing the table, the schema, the indexes. Simple UI.

For users that have no idea about SQL or databases.

Is there anything like that already done and ready to use?

There are lots of apps that do database introspection. Some also generate forms on the fly, but eventually it's necessary to: specify a forms widget for a particular field because SQL schema only describes the data and not the UI; and specify security authorization restrictions on who can create, read, update, or delete data.

And then you want to write arbitrary queries to filter on columns that aren't indexed; but it's really dangerous to allow clients to run arbitrary SQL queries because there basically are no row/object-level database permissions (the application must enforce row-level permissions).

Datasette is a great tool for read-only database introspection and queries of SQLite databases. https://github.com/simonw/datasette

Sandman2 generates a REST API for an arbitrary database. https://github.com/jeffknupp/sandman2

You can generate Django models and then write admin.py files for each model/table that you want to expose in the django.contrib.admin interface.

There are a number of apps for providing a GraphQL API given introspection of a database that occurs at every startup or at runtime; but that doesn't solve for row-level permissions (or web forms)

If you have an OpenAPI spec for the REST API that runs atop The database, you can generate forms ("scaffolding") from the OpenAPI spec and then customize those with form widgets; optionally with something like json-schema.

It's not safe to allow introspected CRUD like e.g. phpMyAdmin for anything but development. If there are no e.g. foreign-key constraints specified in the SQL schema,a blindly-introspected UI very easily results in database corruption due to invalid foreign key references (because the SQL schema doesn't specify what table.column a foreign key references).

Django models, for example, unify SQL schema and forms UI in models.py; admin.py is optional but really useful for scaffolding (such as when you're doing manual testing because you haven't yet written automated tests) https://docs.djangoproject.com/en/2.2/ref/contrib/admin/#mod...

[-]

California approves solar-powered EV charging network and electric school buses

> The press release from the company said, “heavy-duty vehicles produce more particulate matter than all of the state’s power plants combined”.

> […] for instance why only “10 school buses”?

IARC has recognized diesel exhaust as carcinogenic (lung cancer) since 2012.

Are there other electric school bus programs in the US?

(edit)

https://www.trucks.com/2019/03/22/can-electric-school-buses-...

> Most school systems don’t have sufficient capital to finance the high initial costs of electric bus purchases and charging infrastructure development, he said.

> In the U.S., the school bus market is about 33,000 to 35,000 vehicles per year – about six times more than transit buses.

[-]

You May Be Better Off Picking Stocks at Random, Study Finds

[+]

In addition to diversification that reduces risk of overexposure to down sectors or typically over-performing assets, index funds have survivorship bias: underperforming assets are replaced by assets that meet the fund's criteria.

[+]
[-]

Root: CERN's scientific data analysis framework for C++

[+]

> With frameworks like Python pandas, you always end up having to manually partition your data if it doesn’t fit in memory.

"Pandas Docs > Pandas Ecosystem > Out of Core" lists a number of solutions for working with datasets that don't fit into RAM: Blaze, Dask, Dask-ML (dask-distributed; Scikit-Learn, XGBoost, TensorFlow), Koalas, Odo, Ray, Vaex https://pandas-docs.github.io/pandas-docs-travis/ecosystem.h...

The dask API is very similar to the pandas API.

Are there any plans for ROOT to gain support for Apache Parquet, and/or Apache Arrow zero-copy reads and SIMD support, and/or https://RAPIDS.ai (Arrow, numba, Dask, pandas, scikit-learn, XGboost, spark, CUDA-X GPU acceleration, HPC)? https://arrow.apache.org/

https://root.cern.ch/root-has-its-jupyter-kernel (2015)

> Yet another milestone of the integration plan of ROOT with the Jupyter technology has been reached: ROOT now offers a Jupyter kernel! You can try it already now.

> ROOT is the 54th entry in this list and this is pretty cool. Now not only the PyROOT, the ROOT Python bindings, are integrated with notebooks but it's also possible to express your data mining in C++ within a notebook, taking advantage of all the powerful features of ROOT - plotting (now also interactive thanks to (Javascript ROOT](https://root.cern.ch/js/)), multivariate analysis, linear algebra, I/O and reflection: all available within a notebook.

Does this work with JupyterLab now? (edit) Here's the JupyterLab extension developer guide: https://jupyterlab.readthedocs.io/en/stable/developer/extens... (edit) here's the gh issue: https://github.com/root-project/jsroot/issues/166

...

ROOT is now installable with conda: `conda install -c conda-forge root metakernel jupyterlab # notebook`

[+]
[+]
[+]
[+]
[-]

MesaPy: A Memory-Safe Python Implementation based on PyPy (2018)

[+]

> Since then, I’ve found RustPython [0] which is progressing toward feature parity with CPython but entirely written in Rust (!). A side benefit is that it compiles to Web Assembly, so if you could sandbox it without too much extra overhead.

It's now possible to run JupyterLab entirely within a browser with jyve (JupyterLab + pyodide) https://github.com/iodide-project/pyodide/issues/431

Pyodide:

> Pyodide brings the Python runtime to the browser via WebAssembly, along with the Python scientific stack including NumPy, Pandas, Matplotlib, parts of SciPy, and NetworkX. The packages directory lists over 35 packages which are currently available.

Is the RustPython WASM build more performant or otherwise preferable to brython or pyodide?

[-]

Ask HN: Configuration Management for Personal Computer?

Hello HN,

Every couple of years I find myself facing the same old tired routine: migrating my stuff off some laptop or desktop to a new one, usually combined with an OS upgrade. Is there anything like the kind of luxuries we now consider normal on the server side (IaaS; Terraform; maybe Ansible) that can be used to manage your PC and that would make re-imaging it as easy as it is on the server side?

Ansible is worth the extra few minutes, IMHO.

+ (minimal) Bootstrap System playbook

+ Complete System playbook (that references group_vars and host_vars)

+ Per-machine playbooks stored alongside the ansible inventory, group_vars, and host_vars in a separate repo (for machine-specific kernel modules and e.g. touchpad config)

+ User playbook that calls my bootstrap dotfiles shell script

+ Bootstrap dotfiles shell script, which creates symlinks and optionally installs virtualenv+virtualenvwrapper, gitflow and hubflow, and some things with pipsi. https://github.com/westurner/dotfiles/blob/develop/scripts/b...

+ setup_miniconda.sh that creates a CONDA_ROOT and CONDA_ENVS_PATH for each version of CPython (currently py27-py37)

Over the years, I've worked with Bash, Fabric, Puppet, SaltStack, and now Ansible + Bash

I log shell commands with a script called usrlog.sh that creates a $USER and per-virtualenv tab-delimited logfiles with unique per-terminal-session identifiers and ISO8601 timestamps; so it's really easy to just grep for the apt/yum/dnf commands that I ran ad-hoc when I should've just taken a second to create an Ansible role with `ansible-galaxy init ansible-role-name ` and referenced that in a consolidated system playbook with a `when` clause. https://westurner.github.io/dotfiles/usrlog.html#usrlog

A couple weeks ago I added an old i386 netbook to my master Ansible inventory and system playbook and VScode wouldn't install because VScode Linux is x86-64 only and the machine doesn't have enough RAM; so I created when clauses to exclude VScode and extensions on that box (with host_vars). Gvim with my dotvim works great there too though. Someday I'll merge my dotvim with SpaceVim and give SpaceMacs a try; `git clone; make install` works great, but vim-enhanced/vim-full needs to be installed with the system package manager first so that the vimscript plugin installer works and so that the vim binary gets updated when I update all.

I've tested plenty of Ansible server configs with molecule (in docker containers), but haven't yet taken the time to do a full workstation build with e.g. KVM or VirtualBox or write tests with testinfra. It should be easy enough to just run Ansible as a provisioner in a Vagrantfile or a Packer JSON config. VirtualBox supports multi-monitor VMs and makes USB passthrough easy, but lately Docker is enough for everything but Windows (with a PowerShell script that installs NuGet packages with chocolatey) and MacOS (with a few setup scripts that download and install .dmg's and brew) VMs. Someday I'll write or adapt Ansible roles for Windows and Mac, too.

I still configure browser profiles by hand; but it's pretty easy because I just saved all the links in my tools doc: https://westurner.github.io/tools/#browser-extensions

Someday, I'll do bookmarks sync correctly with e.g. Chromium and Firefox; which'll require extending westurner/pbm to support Firefox SQLite or a rewrite in JS with the WebExtension bookmarks API.

A few times, I've decided to write docs for my dotfiles and configuration management policies like someone else is actually going to use them; it seemed like a good exercise at the time, but invariably I have to figure out what the ultimate command sequence was and put that in a shell script (or a Makefile, which adds a dependency on GNU make that's often worth it)

Clonezilla is great and free, but things get out of date fast in a golden master image. It's actually possible to PXE boot clonezilla with Cobbler, but, AFAICT, there's no good way to secure e.g. per-machine disk or other config with PXE. Apt-cacher-ng can proxy-cache-mirror yum repos, too. Pulp requires a bit of RAM but looks like a solid package caching system. I haven't yet tested how well Squid works as a package cache when all of the machines are simultaneously downloading the exact same packages before a canary system (e.g. in a VM) has populated the package cache.

I'm still learning to do as much as possible with Docker containers and Dockerfiles or REES (Reproducible Execution Environment Specifications) -compatible dependency configs that work with e.g. repo2docker and https://mybinder.org/ (BinderHub)

[-]

GitHub Actions now supports CI/CD, free for public repositories

[+]
[+]
[+]
[+]

You can create a separate repo with your own CI config that pulls in the code you want to test; and thus ignore the code's CI config file. When something breaks, you'd then need to determine in which repo something changed: in the CI config repo, or the code repo. And then, you have CI events attached to PRs in the CI config repository.

IMHO it makes sense to have CI config version controlled in the same repo as the code. Unless there's a good tool for bisecting across multiple repos and subrepos?

[-]

The Fed is getting into the Real-Time payments business

apo | 2019-08-05 17:19:30 | 96 | # | ^

This system will need to interface with other domestic and international settlement and payments networks.

There is thus an opportunity for standards, a need for federation, and a need to make it easy for big players to offer liquidity.

As far as I understand, e.g. Ripple and Stellar solve basically exactly the 24x7x365 RTGS problem that FedNow intends to solve; and, they allow all sorts of assets to be plugged into the network. Could FedNow just use a different UNL (Unique Node List) with participating banks operating trusted validators and/or offering liquidity ("liquidity provisioning")?

Notably, Ripple is specifically positioned to do international interbank real time gross settlement (RTGS) and remittances. Ripple could integrate with FedNow directly. Most efficiently, if it complies with KYC/AML requirements, FedNow could operate an XRP Ledger. Or, each bank could operate XRP Ledgers. https://xrpl.org/become-an-xrp-ledger-gateway.html

Getting thousands of banks to comply with an evolving API / EDI spec is no small task. Blockchain solutions require API compliance, have solutions for governance where there are a number of stakeholders seeking to reach consensus, and lack single points of failure.

Here's to hoping that we've learned something about decentralizing distributed systems for resiliency.

>> In contrast, the XRP Ledger requires 80 percent of validators on the entire network, over a two-week period, to continuously support a change before it is applied. Of the approximately 150 validators today, Ripple runs only 10. Unlike Bitcoin and Ethereum — where one miner could have 51 percent of the hashing power — each Ripple validator only has one vote in support of an exchange or ordering a transaction. https://news.ycombinator.com/item?id=19195050

So, you want to get banks onboard with only one s'coin USD stablecoin; but you don't want to deal with exchanges or FOREX or anything because that's a different thing? And, this is not just yet another ACH with lower clearance time?

> Interledger Architecture

https://interledger.org/rfcs/0001-interledger-architecture/

> Interledger provides for secure payments across multiple assets on different ledgers. The architecture consists of a conceptual model for interledger payments, a mechanism for securing payments, and a suite of protocols that implement this design.

> The Interledger Protocol (ILP) is the core of the Interledger protocol suite. Colloquially, the whole Interledger stack is sometimes referred to as "ILP". Technically, however, the Interledger Protocol is only one layer in the stack.

> Interledger is not a blockchain, a token, nor a central service. Interledger is a standard way of bridging financial systems. The Interledger architecture is heavily inspired by the Internet architecture described in RFC 1122, RFC 1123 and RFC 1009.

[...]

> You can envision the Interledger as a graph where the points are individual nodes and the edges are accounts between two parties. Parties with only one account can send or receive through the party on the other side of that account. Parties with two or more accounts are connectors, who can facilitate payments to or from anyone they're connected to.

> Connectors provide a service of forwarding packets and relaying money, and they take on some risk when they do so. In exchange, connectors can charge fees and derive a profit from these services. In the open network of the Interledger, connectors are expected to compete among one another to offer the best balance of speed, reliability, coverage, and cost.

Why should we prefer an immutable, cryptographically-signed blockchain solution over SQL/BigTable/MQ for FedNow?

Blockchain and payments standards: https://news.ycombinator.com/item?id=19813340

... Here's the notice and request for comment PDF: "Docket No. OP – 1670: Federal Reserve Actions to Support Interbank Settlement of Faster Payments" https://www.federalreserve.gov/newsevents/pressreleases/file...

"Federal Reserve announces plan to develop a new round-the-clock real-time payment and settlement service to support faster payments" https://www.federalreserve.gov/newsevents/pressreleases/othe...

[-]

A Giant Asteroid of Gold Won’t Make Us Richer

> this example shows that real wealth doesn’t actually come from golden hoards. It comes from the productive activities of human beings creating things that other human beings desire.

Value, Price, and Wealth

[+]

Good call. I don't know where I was going with that. Cost, price, value, and wealth.

Are there better examples for illustrating the differences between these kind of distinct terms?

Less convertible collectibles like coins and baseball cards (that require energy for exchange) have (over time t): costs of production, marketing, and distribution; retail sales price; market price; and 'value' which is abstract relative (opportunity cost in terms of fiat currency (which is somehow distinct from price at time t (possibly due to 'speculative information')))

Wealth comes from relationships, margins between costs and prices, long term planning, […]

[+]
[-]

Abusing the PHP Query String Parser to Bypass IDS, IPS, and WAF

[+]
[+]

Possible solutions:

(1) Change all underscores in WAF rule URL attribute names to the appropriate non-greedy regex. Though I'm not sure about the regex the article suggests: '.' only matches one character, AFAIU.

(2) Add a config parameter to PHP that turns off the magical url parameter name mangling that no webapp should ever depend on ( and have it default to off because if you rely on this 'feature' you should have to change a setting in php.ini anyway )

[-]

Ask HN: Scripts/commands for extracting URL article text? (links -dump but)

I'd like to have a Unix script that basically generates a text file named, with the page title, with the article text neatly formatted.

This seems to me to be something that would be so commonly desired by people that it would've been done and done and done a hundred times over by now, but I haven't found the magic search terms to dig up people's creations.

I imagine it starts with "links -dump", but then there's using the title as the filename, and removing the padded left margin, wrapping the text, and removing all the excess linkage.

I'm a beginner-amateur when it comes to shell scripting, python, etc. - I can Google well and usually understand script or program logic but don't have terms memorized.

Is this exotic enough that people haven't done it, or as I suspect does this already exist and I'm just not finding it? Much obliged for any help.

[+]

There could be collisions where `fname2` is the same for different pages; resulting in unintentionally overwriting. A couple possible solutions: generate a random string and append it to the filename, set fname2 to a hash of the URL, replace unsafe filename characters like '/' and/or '\' and/or '\n' with e.g. underscores. IIRC, URLs can be longer than the max filename length of many filesystems, so hashes as filenames are the safest solution. You can generate an index of the fetched URLs and store it with JSON or e.g. SQLite (with Records and/or SQLAlchemy, for example).

If or when you want to parallelize (to do multiple requests at once because most of the time is spent waiting for responses from the network) write-contention for the index may be an issue that SQLite solves for better than a flatfile locking mechanism like creating and deleting an index.json.lock. requests3 and aiohttp-requests support asyncio. requests3 supports HTTP/2 and connection pooling.

SQLite can probably handle storing the text of as many pages as you throw at it with the added benefit of full-text search. Datasette is a really cool interface for sqlite databases of all sorts. https://datasette.readthedocs.io/en/stable/ecosystem.html#to...

...

Apache Nutch + ElasticSearch / Lucene / Solr are production-proven crawling and search applications: https://en.m.wikipedia.org/wiki/Apache_Nutch

> I imagine it starts with "links -dump", but then there's using the title as the filename,

The title tag may exceed the filename length limit, be the same for nested pages, or contain newlines that must be escaped.

These might be helpful for your use case:

"Newspaper3k: Article scraping & curation" https://github.com/codelucas/newspaper

lazyNLP "Library to scrape and clean web pages to create massive datasets" https://github.com/chiphuyen/lazynlp/blob/master/README.md#s...

scrapinghub/extruct https://github.com/scrapinghub/extruct

> extruct is a library for extracting embedded metadata from HTML markup.

> It also has a built-in HTTP server to test its output as JSON.

> Currently, extruct supports:

> - W3C's HTML Microdata

> - embedded JSON-LD

> - Microformat via mf2py

> - Facebook's Open Graph

> - (experimental) RDFa via rdflib

[-]

NPR's Guide to Hypothesis-Driven Design for Editorial Projects

HDD – Hypothesis-Driven Development – Research, Plan, Prototype, Develop, Launch, Review.

The article lists (and links to!) "Lean UX" [1] and Google Ventures' Design Sprint Methodology as inspirations.

[1] "Lean UX: Applying Lean Principles to Improve User Experience" http://shop.oreilly.com/product/0636920021827.do

[2] https://www.gv.com/sprint/

"How To Write A Technical Paper" [3][4] has: (Related Work, System Model, Problem Statement), (Your Solution), (Analysis), (Simulation, Experimentation), (Conclusion)

[3] https://news.ycombinator.com/item?id=18226543

[4] https://westurner.github.io/hnlog/#story-18225197

[-]

Gryphon: An open-source framework for algorithmic trading in cryptocurrency

reso | 2019-06-20 14:56:56 | 236 | # | ^
[+]

> As far as I know there isn't anything out there like this, in any market (not just cryptocurrencies).

How does Gryphon compare to Catalyst (Zipline)? https://github.com/enigmampc/catalyst

They list a few example algorithms: https://enigma.co/catalyst/example-algos.html

"Ask HN: Why would anyone share trading algorithms and compare by performance?" https://news.ycombinator.com/item?id=15802834 (pyfolio, popular [Zipline] algos shared through Quantopian)

"Superalgos and the Trading Singularity" https://news.ycombinator.com/item?id=19109333 (awesome-quant,)

[+]
[+]

Would CCXT be useful here? https://github.com/ccxt/ccxt

> The ccxt library currently supports the following 135 cryptocurrency exchange markets and trading APIs:

[+]
[+]
[+]
[+]
[+]

> In any case Gryphon uses Cython to compile itself down to C, which isn't quite as good as writing in native C but is a good chunk of the way there.

Would there be any advantage to asyncio with uvloop (also written in Cython (on libuv like Node) like Pandas)? https://github.com/MagicStack/uvloop

IDK how many e.g. signals routines benefit from asyncio yet.

[+]
[+]
[+]
[+]

Whether there's anything like an equilibrium in cryptoasset markets where there are no underlying fundamentals is debatable. While there's no book price, PoW coin prices might be rationally describable in terms of (average_estimated cost of energy + cost per GH/s + 'speculative value')

A proxy for energy costs, chip costs, and speculative information

Are there standard symbols for this?

Can cryptoasset market returns be predicted with quantum harmonic oscillators as well? What NN topology can learn a quantum harmonic model? https://news.ycombinator.com/item?id=19214650

"The Carbon Footprint of Bitcoin" (2019) defines a number of symbols that could be standard in [crypto]economics texts. Figure 2 shows the "profitable efficiency" (which says nothing of investor confidence and speculative information and how we maybe overvalue teh security (in 2007-2009)). Figure 5 lists upper and lower estimates for the BTC network's electricity use. https://www.cell.com/joule/fulltext/S2542-4351(19)30255-7

Here's a cautionary dialogue about correlative and causal models that may also be relevant to a cryptoasset price NN learning experiment: https://news.ycombinator.com/item?id=20163734

[-]

Wind-Powered Car Travels Downwind Faster Than the Wind

> The unusual wind-powered car hit a top speed 2.86 times faster than the wind during one recent run,

I can't even.

[+]
[+]
[-]

NOAA upgrades the U.S. global weather forecast model

> Working with other scientists, Lin developed a model to represent how flowing air carries these substances. The new model divided the atmosphere into cells or boxes and used computer code based on the laws of physics to simulate how air and chemical substances move through each cell and around the globe.

> The model paid close attention to conserving energy, mass and momentum in the atmosphere in each box. This precision resulted in dramatic improvements in the accuracy and realism of the atmospheric chemistry.

Global Forecast System > Future https://en.wikipedia.org/wiki/Global_Forecast_System#Future

[-]

A plan to change how Harvard teaches economics

[+]
[+]
[+]
[+]
[+]

> apologists for the continuation of rent-seeking policies that entrench the rich and mighty.

This.

"THE IMF CONFIRMS THAT 'TRICKLE-DOWN' ECONOMICS IS, INDEED, A JOKE" https://psmag.com/economics/trickle-down-economics-is-indeed...

> INCREASING THE INCOME SHARE TO THE BOTTOM 20 PERCENT OF CITIZENS BY A MERE ONE PERCENT RESULTS IN A 0.38 PERCENTAGE POINT JUMP IN GDP GROWTH.

> The IMF report, authored by five economists, presents a scathing rejection of the trickle-down approach, arguing that the monetary philosophy has been used as a justification for growing income inequality over the past several decades. "Income distribution matters for growth," they write. "Specifically, if the income share of the top 20 percent increases, then GDP growth actually declined over the medium term, suggesting that the benefits do not trickle down."

"Causes and Consequences of Income Inequality: A Global Perspective" (2015) https://scholar.google.com/scholar?hl=en&as_sdt=0%2C43&q=%22...

I'll add that we tend to overlook the level of government spending during periods trickle-down economics and confound. Change in government spending (somewhat unfortunately regardless of revenues) is a relevant factor.

Let's make this economy great again? How about you identify the decade(s) you're referring to and I'll show you the tax revenue (on income and now capital gains), federal debt per capital, and the growth in GDP.

[+]

> All this is to say that, while data is useful for validation, it is not useful for prediction. The last thing we need is a black-box machine learning model to make major economic decisions off of. What we do need is proper models that are then validated, which don't necessarily need 'big data.'

Hand-wavy theory - predicated upon physical-world models of equillibrium which are themselves classical and incomplete - without validation is preferable to empirical models? Please.

Estimating the predictive power of some LaTeX equations is a different task than measuring error of a trained model.

If the model does not fit all of the big data, the error term is higher; regardless of whether the model was pulled out of a hat in front of a captive audience or deduced though inference from actual data fed through an unbiased analysis pipeline.

If the 'black-box predictive model' has lower error for all available data, the task is then to reverse the model! Not to argue for unvalidated theory.

Here are a few discussions regarding validating economic models, some excellent open econometric lectures (as notebooks that are unfortunately not in an easily-testable programmatic form), the lack of responsible validation, and some tools and datasets that may be useful for validating hand-wavy classical economic theories:

"When does the concept of equilibrium work in economics?" https://news.ycombinator.com/item?id=19214650

> "Lectures in Quantitative Economics as Python and Julia Notebooks" https://news.ycombinator.com/item?id=19083479 (data sources (pandas-datareader, pandaSDMX), tools, latex2sympy)

That's just an equation in a PDF.

(edit) Here's another useful thread: "Ask HN: Data analysis workflow?" https://news.ycombinator.com/item?id=18798244

[+]

Backtesting algorithmic trading algorithms is fairly simple: what actions would the model have taken given the available data at that time, and how would those trading decisions have affected the single objective dependent variable. Backtesting, paper trading, live trading.

Medicine (and also social sciences) is indeed more complex; but classification and prediction are still the basis for making treatment recommendations, for example.

Still, the task really is the same. A NN (like those that Torch, Theano, TensorFlow, and PyTorch produce; now with the ONNX standard for neural network model interchange) learns complex relations and really doesn't care about causality: minimize the error term. Recent progress in reducing the size of NN models e.g. for offline natural language classification on mobile devices has centered around identifying redundant neuronal connections ("from 100GB to just 0.5GB"). Reversing a NN into a far less complex symbolic model (with variable names) is not a new objective. NNs are being applied for feature selection, XGBoost wins many Kaggle competitions, and combinations thereof appear to be promising.

Actually testing second-order effects of evidence-based economic policy recommendations is certainly a complex highly-multivariate task (with unfortunate ideological digression that presumes a higher-order understanding based upon seeming truisms that are not at all validated given, in many instances, any data). A causal model may not be necessary or even reasonably explainable; and what objective dependent variables should we optimize for? Short term growth or long-term prosperity with environmental sustainability?

... "Please highly weight voluntary sustainability reporting metrics along with fundamentals" when making investments and policy decisions?

Were/are the World3 models causal? Many of their predictions have subsequently been validated. Are those policy recommendations (e.g. in "The Limits to Growth") even more applicable today, or do we need to add more labeled data and "Restart and Run All"?

...

From https://research.stlouisfed.org/useraccount/fredcast/faq/ :

> FREDcast™ is an interactive forecasting game in which players make forecasts for four economic releases: GDP, inflation, employment, and unemployment. All forecasts are for the current month—or current quarter in the case of GDP. Forecasts must be submitted by the 20th of the current month. For real GDP growth, players submit a forecast for current-quarter GDP each month during the current quarter. Forecasts for each of the four variables are scored for accuracy, and a total monthly score is obtained from these scores. Scores for each monthly forecast are based on the magnitude of the forecast error. These monthly scores are weighted over time and accumulated to give an overall performance.

> Higher scores reflect greater accuracy over time. Past months' performances are downweighted so that more-recent performance plays a larger part in the scoring.

The #GobalGoals Targets and Indicators may be our best set of variables to optimize for from 2015 through 2030; I suppose all of them are economic.

[+]

Yes, some combination of variables/features grouped and connected with operators that correlate to an optima (some of which are parameters we can specify) that occurs immediately or after a period of lag during which other variables of the given complex system are dangerously assumed to remain constant.

> In fact, this is exactly the blindness that led to people missing the financial crisis

ML was not necessary to recognize the yield curve inversion as a strongly predictive signal correlating to subsequent contraction.

An NN can certainly learn to predict according to the presence or magnitude of a yield curve inversion and which combinations of other features.

- [ ] Exercise: Learning this and other predictive signals by cherry-picking data and hand-optimizing features may be an extremely appropriate exercise.

"This field is different because it's nonlinear, very complex, there are unquantified and/or uncollected human factors, and temporal"

Maybe we're not in agreement about whether AI and ML can do causal inference just as well if not better than humans manipulating symbols with human cognition and physical world intuition. The time is nigh!

In general, while skepticism and caution are appropriate, many fields suffer from a degree of hubris which prevents them from truly embracing stronger AI in their problem domain. (A human person cannot mutate symbol trees and validate with shuffled and split test data all night long)

> Anyone trying to understand economic phenomena needs to be keenly aware of how inference can be done, which requires an understanding (or an approach to) - that is, a theory - of the underlying mechanisms.

I read this as "must be biased by the literature and willing to disregard an unacceptable error term"; but also caution against rationalizing blind findings which can easily be rationalized as logical due to any number of cognitive biases.

Compared to AI, we're not too rigorous about inductive or deductive inference; we simply store generalizations about human behavior and predict according to syntheses of activations in our human NNs.

If you're suggesting that the information theory that underlies AI and ML is insufficient to learn what we humans have learned in a few hundred years of observing and attempting to optimize, I must disagree (regardless of the hardness or softness of the given complex field). Beyond a few combinations/scenarios, our puny little brains are no match for our department's new willing AI scientist.

[+]

> AI, ML and stats will merge, if they haven't already. The distinction will disappear. I believe the issues will not.

All tools are misapplied; including economics professionals and their advice.

Here's a beautiful Venn diagram of "Colliding Web Sciences" which includes economics as a partially independent category: https://www.google.com/search?q=colliding+web+sciences&tbm=i...

A causal model is a predictive model. We must validate the error of a causal model.

Why are theoretic models hand-wavy? "That's just because noise, the model is correct." No, such a model is insufficient to predict changes in dependent variables when in the presence of noise; which is always the case. How does validating a causal model differ from validating a predictive model with historical and future data?

Yield-curve inversion as a signal can be learned by human and artificial NNs. Period. There are a few false positives in historical data: indeed, describe the variance due to "noise" by searching for additional causal and correlative relations in additional datasets.

I searched for "python causal inference" and found a few resources on the first page of search results: https://www.google.com/search?q=python+causal+inference

CausalInference: https://pypi.org/project/CausalInference/

DoWhy: https://github.com/microsoft/dowhy

CausalImpact (Python port of the R package): https://github.com/dafiti/causalimpact

"What is the best Python package for causal inference?" https://www.quora.com/What-is-the-best-Python-package-for-ca...

Search: graphical model "information theory" [causal] https://www.google.com/search?q=graphical+model+%22informati...

Search: opencog causal inference https://www.google.com/search?q=opencog+causal+inference (MOSES, PLN,)

If you were to write a pseudocode algorithm for an econometric researcher's process of causal inference (and also their cognitive processes (as executed in a NN with a topology)), how would that read?

(Edit) Something about the sufficiency of RL (Reinforcement Learning) for controlling cybernetic systems. https://en.wikipedia.org/wiki/Cybernetics

[+]

> What's the point of dumping a bunch of Google results here? At least half the results are about implementations of pretty traditional etatistical / econometric inference techniques.

Here are some tools for causal inference (and a process for finding projects to contribute to instead of arguing about insufficiency of AI/ML for our very special problem domain here). At least one AGI implementation doesn't need to do causal inference in order to predict the outcomes of actions in a noisy field.

Weather forecasting models don't / don't need to do causal inference.

> A/B testing

Is multi-armed bandit feasible for the domain? Or, in practice, are there too many concurrent changes in variables to have any sort of a controlled experiment. Then, aren't you trying to do causal inference with mostly observational data.

> I really don't see how a RL would help with any of this. Care to come up with something concrete?

The practice of developing models and continuing on with them when they seem to fit and citations or impact reinforce is very much entirely an exercise in RL. This is a control system with a feedback loop. A "Cybernetic system". It's not unique. It's not too hard for symbolic or neural AI/ML. Stronger AI can or could do [causal] inference.

[+]
[+]

> By extension, it is impossible for any ML mechanism to predict unobserved interventions without being a causal model.

In lieu of a causal model, when I ask an economist what they think is going to happen and they aren't aware of any historical data - there is no observational data collected following the given combination of variables we'd call an event or an intervention - is it causal inference that they're doing in their head? (With their NN)

> Now, you and me, we can both agree that your model with yield curves is good enough.

Yield curves alone are insufficient due to the rate of false positives. (See: ROC curves for model evalutation just like everyone else)

> We could even agree that you would have found it before the financial crashes,

The given signal was disregarded as a false positive by the appointed individuals at the time; why?

> Some alien that has been analyzing financial systems all across the universe may disagree,

You're going to run out of clean water and energy, and people will be willing to pay for unhealthy sugar water and energy-inefficient transaction networks with a perception of greater security.

That we need Martian scientist as an approach is, IMHO, necessary because of our learned biases; where we've inferred relations that have been reinforced which cloud our assessment of new and novel solutions.

> Such is the difficulty of causal analysis.

What a helpful discussion. Thanks for explaining all of this to me.

Now, I need to go write my own definitions for counterfactual and DGP and include graphical models in there somewhere.

[+]
[+]
[+]

How can you possibly be arguing that we should not be testing models with all available data?

All models are limited by the data they're trained from; regardless of whether they are derived through rigorous, standardized, unbiased analysis or though laudable divine inspiration.

From https://news.ycombinator.com/item?id=19084622 :

> pandas-datareader can pull data from e.g. FRED, Eurostat, Quandl, World Bank: https://pandas-datareader.readthedocs.io/en/latest/remote_da...

> pandaSDMX can pull SDMX data from e.g. ECB, Eurostat, ILO, IMF, OECD, UNSD, UNESCO, World Bank; with requests-cache for caching data requests: https://pandasdmx.readthedocs.io/en/latest/#supported-data-p...

[+]

> To get out of this we have to consider not only what people have done in the past but how they are likely to respond to a given policy change, for which we have no historical data prior to when the policy is enacted, and so we need to make those predictions based on logic in addition to data or we go astray.

"Pete, it's a fool who looks for logic in the chambers of the human heart."

Logically, we might have said "prohibition will reduce substance abuse harms" but the actual data indicates that margins increased. Then, we look at the success of Portugal's decriminalization efforts and cannot at all validate our logical models.

Similarly, we might've logically claimed that "deregulation of the financial industry will help everyone" or "lowering taxes will help everyone" and the data does not support.

So, while I share the concerns about Responsible AI and encoding biases (and second-order effects of making policy recommendations according to non-causal models without critically, logically thinking first) I am very skeptical about our ability to deduce causal relations without e.g. blind, randomized, longitudinal, interventional studies (which are unfortunately basically impossible to do with [economic] policy because there is no "ceteris paribus")

https://personalmba.com/second-order-effects/

"Causal Inference Book" https://news.ycombinator.com/item?id=17504366

> https://www.hsph.harvard.edu/miguel-hernan/causal-inference-...

> Causal inference (Causal reasoning) https://en.wikipedia.org/wiki/Causal_inference ( https://en.wikipedia.org/wiki/Causal_reasoning )

[+]

> If you think prohibition will reduce substance abuse but then you try it and it doesn't, well, you were wrong, so end prohibition.

Maybe we're at a local optima, though. Maybe this is a sign that we should just double down, surge on in there and get the job done by continuing to do the same thing and expecting different results. Maybe it's not the spec but the implementation.

Recommend a play according to all available data, and logic.

> This is also a strong argument for "laboratories of democracy" and local control -- if everybody agrees what to do then there is no dispute, but if they don't then let each local region have their own choice, and then we get to see what happens. It allows more experiments to be run at once. Then in the worst case the damage of doing the wrong thing is limited to a smaller area than having the same wrong policy be set nationally or internationally, and in the best case different choices are good in different ways and we get more local diversity.

"Adjusting for other factors," the analysis began.

- [ ] Exercise / procedure to be coded: Brainstorm and identify [non-independent] features that may create a more predictive model (a model with a lower error term). Search for confounding variables outside of the given data.

[-]

The New York Times course to teach its reporters data skills is now open-source

It's more work to verify all formulas that reference unnamed variables in a spreadsheet than to review the code inputs and outputs in a notebook.

"Teaching Pandas and Jupyter to Northwestern journalism students" [in DC] https://www.californiacivicdata.org/2017/06/07/dc-python-not...

> http://www.firstpythonnotebook.org/

You can also develop d3.js visualizations — just like NYT — with jupyter notebooks and whichever language(s).

"Data-Driven Journalism" ("ddj") https://en.wikipedia.org/wiki/Data-driven_journalism

http://datadrivenjournalism.net/

"The Data Journalism Handbook 1" https://datajournalism.com/read/handbook/one

"The Data Journalism Handbook 2" https://datajournalism.com/read/handbook/two

While there are a number of ScholarlyArticle journals that can publish notebooks, I'm not aware of any newspapers that are prepared to publish notebooks as NewsArticles. It's pretty easy to `jupyter convert --to html` and `--to markdown` or just 'Save as'

Regarding expressing facts as verifiable claims with structured data in HTML and/or blockchains: "Fact Checks" https://news.ycombinator.com/item?id=15529140

Does this course recommend linking to every source dataset and/or including full citations (with DOI) in the article? Does this course recommend getting a free DOI for the published revision of an e.g. GitHub project repository (containing data, and notebooks and/or the article text) with Zenodo?

[-]

No Kings: How Do You Make Good Decisions Efficiently in a Flat Organization?

Group decision-making > Formal systems: https://en.wikipedia.org/wiki/Group_decision-making#Formal_s...

> Consensus decision-making, Voting-based methods, Delphi method, Dotmocracy

Consensus decision-making: https://en.wikipedia.org/wiki/Consensus_decision-making

There's a field that some people are all calling "Collaboration Engineering". I learned about this from a university course in Collaboration.

6 Patterns of Collaboration [GRCOEB] — Generate, Reduce, Clarify, Organize, Evaluate, Build Consensus

7 Layers of Collaboration [GPrAPTeToS] — Goals, Products, Activities, Patterns of Collaboration, Techniques, Tools, Scripts

The group decision making processes described in the article may already be defined with the thinkLets design pattern language.

A person could argue against humming for various unspecified reasons.

I'll just CC this here from my notes, which everyone can read here [1]:

“Collaboration Engineering: Foundations and Opportunities” de Vreede (2009) http://aisel.aisnet.org/jais/vol10/iss3/7/

“A Seven-Layer Model of Collaboration: Separation of Concerns for Designers of Collaboration Systems” Briggs (2009) http://aisel.aisnet.org/icis2009/26/

Six Patterns of Collaboration “Defining Key Concepts for Collaboration Engineering” Briggs (2006) http://aisel.aisnet.org/amcis2006/17/

“ThinkLets: Achieving Predictable, Repeatable Patterns of Group Interaction with Group Support Systems (GSS)” http://www.academia.edu/259943/ThinkLets_Achieving_Predictab...

https://scholar.google.com/scholar?q=thinklets

[1] https://wrdrd.github.io/docs/consulting/team-building#collab...

[-]

4 Years of College, $0 in Debt: How Some Countries Make Education Affordable

It at least makes sense to pay for doctors and nurses to go to school, right? If you want to care for others and you do the work to earn satisfactory grades, I think that investing in your education would have positive ROI.

We had plans here in the US to pay for two years of community college for whoever ("America's College Promise"). IDK what happened to that? We should have called it #ObamaCollege so that everyone could attack corporate welfare and bad investments with no ROI.

New York has the Excelsior scholarship for CUNY and SUNY. Tennessee pays for college with lottery proceeds. Are there other state-level efforts to fund higher education in the US such that students can finish school debt-free or close to it?

There are MOOCs (online courses) which are worth credit hours for the percentage of people that commit to finishing the course. https://www.classcentral.com/

Khan Academy has free SAT, MCAT, NCLEX-RN, GMAT, and LSAT test prep and primary and supplementary learning resources. https://www.khanacademy.org/test-prep

Free education: https://en.wikipedia.org/wiki/Free_education

[-]

Ask HN: What jobs can a software engineer take to tackle climate change?

I'm a software engineer with a diverse background in backend, frontend development.

How do I find jobs related to tackling global warming and climate change in Europe for an English speaker?

Open to ideas and thoughts.

> I'm a software engineer with a diverse background in backend, frontend development.

> How do I find jobs related to tackling global warming and climate change in Europe for an English speaker?

While not directly answering the question, here are some ideas for purchasing, donating, creating new positions, and hiring people that care:

Write more efficient code. Write more efficient compilers. Optimize interpretation and compilation so that the code written by people with domain knowledge who aren't that great at programming who are trying to solve other important problems is more efficient.

Push for PPAs (Power Purchase Agreements) that offset energy use. Push for directly sourcing clean energy.

Use services that at least have 100% PPAs for the energy they use: services that run on clean energy sources.

Choose green datacenters.

- [ ] Add the capability for cloud resource schedulers like Kubernetes and Terraform to prefer or require clean energy datacenters.

Choose to work with companies that voluntarily choose to do sustainability reporting.

Work to help develop (and popularize) blockchain solutions that are more energy efficient and that have equal or better security assurances as less efficient chains.

Advocate for clean energy. Donate to NGOs working for our environment and for clean energy.

Invest in clean energy. There are a number of clean energy ETFs, for example. Better energy storage is a good investment.

Push for certified green buildings and datacenters.

- [ ] We should create some sort of a badge and structured data (JSONLD, RDFa, Microdata) for site headers and/or footers that lets consumers know that we're working toward '200% green' so that we can vote with our money.

Do not vote for people who are rolling back regulations that protect our environment. Pay an organization that pays lobbyists to work the system: that's the game.

Help explain why it's both environment-rational and cost-rational to align with national and international environmental sustainability and clean energy objectives.

Argue that we should make external costs internal in order that markets will optimize for what we actually want.

Thermodynamics is part of the physics curriculum for many software engineering and computer science degrees.

There are a number of existing solutions that solve for energy inefficiency due to unreclaimed waste heat.

"Thermodynamics of Computation Wiki" https://news.ycombinator.com/item?id=18146854

"Why Do Computers Use So Much Energy?" https://news.ycombinator.com/item?id=18139654

[-]

YC's request for startups: Government 2.0

There's money to be earned in solving for the #GlobalGoals Goals, Targets, and Indicators:

The Global Goals

1. No Poverty

2. Zero Hunger

3. Good Health & Well-Being

4. Quality Education

5. Gender Equality

6. Clean Water & Sanitation

7. Affordable & Clean Energy

8. Decent Work & Economic Growth

9. Industry, Innovation & Infrastructure

10. Reduced Inequalities

11. Sustainable Cities and Communities

12. Responsible Consumption & Production

13. Climate Action

14. Life Below Water

15. Life on Land

16. Peace and Justice & Strong Institutions

17. Partnerships for the Goals

https://en.wikipedia.org/wiki/Sustainable_Development_Goals

[-]

Almost 40% of Americans Would Struggle to Cover a $400 Emergency

[+]

> I always wonder what proportion of that group is due to insufficient income

According to the Social Security Administration [1]:

2017 Average net compensation: 48,251.57

2017 Median net compensation: 31,561.49

The FPL (Federal Poverty Level) income numbers for Medicaid and the Children's Health Insurance Program (CHIP) eligibility [2]:

>> $12,140 for individuals, $16,460 for a family of 2, $20,780 for a family of 3, $25,100 for a family of 4, $29,420 for a family of 5, $33,740 for a family of 6, $38,060 for a family of 7, $42,380 for a family of 8

Wages are not keeping up with corporate profits. That can't all be due to automation.

The minimum wage is only one factor linked to price inflation. We can raise wages and still keep inflation down to an ideal range.

Maybe it's that we don't understand what it's like to live on $12K or $32K a year (without healthcare due to lack of Medicaid expansion; due to our collective failure to instill charity as a virtue and getting people back on their feet as a good investment). How could we learn (or remember!) about what it's like to be in this position (without zero-interest bank loans to bail us out)?

> and what proportion is due to terrible financial literacy.

The r/personalfinance wiki is one good resource for personal finance. From [3]:

>> Personal Finance (budgets, interest, growth, inflation, retirement)

Personal Finance https://en.wikipedia.org/wiki/Personal_finance

Khan Academy > College, careers, and more > Personal finance https://www.khanacademy.org/college-careers-more/personal-fi...

"CS 007: Personal Finance For Engineers" https://cs007.blog

https://reddit.com/r/personalfinance/wiki

... How can we make personal finance a required middle and high school curriculum component? [4]

"What are some ways that you can save money in order to meet or exceed inflation?"

Dave Ramsey's 7 Baby Steps to financial freedom [5] seem like good advice? Is the debt snowball method ideal for minimizing interest payments?

[1] https://www.ssa.gov/OACT/COLA/central.html

[2] https://www.healthcare.gov/glossary/federal-poverty-level-fp...

[3] "Ask HN: How can you save money while living on poverty level?" https://news.ycombinator.com/item?id=18894582

[4] "Consumer science (a.k.a. home economics) as a college major" https://news.ycombinator.com/item?id=17894632

[5] https://www.daveramsey.com/dave-ramsey-7-baby-steps

[-]

Congress should grow the Digital Services budget, it more than pays for itself

> The U.S. Digital Service isn’t perfect, but it is clearly working. The team estimates that for every $1 million invested in USDS that the government will avoid spending $5 million and save thousands of labor hours. Over a five-year period, the team’s efforts will save $1.1 billion, redirect almost 2,000 labor years towards higher value work, and generate over 400 percent return on investment. Most importantly, USDS will continue to deliver better government services for the American people, including Veterans who deserve better.

> In the private sector, these kinds of numbers would not lead to a 50 percent cut in budget. Instead, you’d clearly invest further with that kind of return. Considering the ambitious goals set out in the President’s Management Agenda, the Trump Administration should double down on better support for the public, our troops, and our veterans. The best way to do that is clearly through investments like USDS.

Why would you halve the budget of a team that's yielding a more than 400% ROI (in terms of cost savings)?

https://en.wikipedia.org/wiki/United_States_Digital_Service

[+]

USDS reports 400% ROI in savings to the taxpayers who fund the government with tax revenue (instead of kicking the can down the road with debt financing) and improvements in customer service quality.

https://www.usaspending.gov (Federal Funding Accountability and Transparency Act of 2006 (Obama, McCain, Carper, Coburn)) has more fine-grained spending data, but not credit-free immutable distributed ledger transaction IDs, quantitative ROI stats, or performance.gov and #globalgoals goal alignment. We'd need a metadata field on spending bills to link to performance.gov and SDG Goals, Targets, and Indicators.

"Transparency and Accountability"

IIRC, here on HN, I've mentioned a number of times -- and quoted in the full from -- the 13 plays of the USDS Digital Services Playbook; all of which are applicable to and should probably be required reading for all government IT and govtech: https://playbook.cio.gov/

There are forms with workflow states that need human review sometimes. USDS helps with getting those processes online in order to reduce costs, increase cost-efficiency, and increase quality of service.

The Trillion-Dollar Annual Interest Payment

> Given the recent actions of Congress, and the years of prior inaction in changing the nation’s fiscal path, the U.S. government’s annual interest payment will eclipse annual defense spending in only six years. By 2025, annual interest costs on the national debt will reach $724 billion, while annual defense spending will reach $706 billion. To put that into perspective, in the 2018 fiscal year, the U.S. government spent $325 billion in interest payments and spent $622 billion in defense (Exhibit 2).

Why would you cut taxes and debt finance our nation's future?

[-]

Oak, a Free and Open Certificate Transparency Log

[+]
[+]

> Great use case for blockchain technology

>> CT logs are already chained

Trillian is a centralized Merkle tree: it doesn't support native replication (AFAIU?) and there is a still a password that can delete or recreate the chain (though we can track for any such inappropriate or errant modifications (due to e.g. solar flares) by manually replicating and verifying every entry in the chain, or trusting that everything before whatever we consider to be a known hash (that could be colliding) is unmodified (since the last time we never verified those entries)).

According to the trillian README, trillian depends upon MySQL/MariaDB and thus internal/private replication is as good as the SQL replication model (which doesn't have a distributed consensus algorithm like e.g. paxos).

A Merkle tree alone is not a blockchain; though it provides more assurance of data integrity than a regular tree, verifying that the whole chain of hashes actually is good and distributed replication without configuring e.g. SSL certs are primary features of blockchains.

[+]

Which components of the system are we discussing?

PKI is necessarily centralized: certs depend upon CA certs which can depend upon CA certs. If any CA is compromised (e.g. by theft or brute force (which is inestimably infeasible given current ASIC resources' preference for legit income)) that CA can sign any CRL. A CT log and a CT log verifier can help us discover that a redundant and so possibly unauthorized cert has been issued for a given domain listed in an x.509 cert CN/SAN.

The CT log itself - trillian, for Google and now LetsEncrypt, too - though, runs on MySQL; which has one root password.

The system of multiple independent, redundant CT logs is built upon databases that depend upon presumably manually configured replication keys.

Does my browser call a remote log verifier API over (hopefully pinned with a better fingerprint than MD5) HTTPS?

[+]

Centralized and decentralized are overloaded terms. We could argue that every system that depends upon DNS is a centralized (and thus has a single point of failure).

We could describe replication models as centralized or decentralized. Master/master SQL replication is still not decentralized (regardless of whether there are multiple A records or multiple static IPs configured in the client).

With PKI, we choose the convenience of trusting a CA bundle over having to manually check every cert fingerprint.

Whether a particular chain is centralized or decentralized is often bandied about. When there are a few mining pools that effectively choose which changes are accepted, that's not decentralized either.

That there are multiple redundant independent CT logs is a good thing.

How do I, as a concerned user, securely download (and securely mirror?) one or all of the CT logs and verify that none of the record hashes don't depend upon the previous hash? If the browser relies upon a centralized API for checking hash fingerprints, how is that decentralized?

[+]
[-]

Death rates from energy production per TWh

Apparently the deaths are justified because energy.

Are the subsidies and taxes (incentives and penalties) rational in light of the relative harms of each form of energy?

"Study: U.S. Fossil Fuel Subsidies Exceed Pentagon Spending" https://www.rollingstone.com/politics/politics-news/fossil-f...

> The IMF found that direct and indirect subsidies for coal, oil and gas in the U.S. reached $649 billion in 2015. Pentagon spending that same year was $599 billion.

> The study defines “subsidy” very broadly, as many economists do. It accounts for the “differences between actual consumer fuel prices and how much consumers would pay if prices fully reflected supply costs plus the taxes needed to reflect environmental costs” and other damage, including premature deaths from air pollution.

IDK whether they've included the costs of responding to requests for help with natural disasters that are more probable due to climate change caused by these "externalties" / "external costs" of fossil fuels.

[+]

Why isn't the market choosing the least harmful, least lethal energy sources? Energy is for the most part entirely substitutable: switching costs for consumers like hospitals are basically zero.

(Everyone is free to invest in clean energy at any time)

[+]

100% Renewable Energy https://en.wikipedia.org/wiki/100%25_renewable_energy

> The main barriers to the widespread implementation of large-scale renewable energy and low-carbon energy strategies are political rather than technological. According to the 2013 Post Carbon Pathways report, which reviewed many international studies, the key roadblocks are: climate change denial, the fossil fuels lobby, political inaction, unsustainable energy consumption, outdated energy infrastructure, and financial constraints.

We need to make the external costs of energy production internal in order to create incentives to prevent these fossil fuel deaths and other costs.

[-]

Use links not keys to represent relationships in APIs

A thing may be identified by a URI (/person/123) for which there are zero or more URL routes (/person/123, /v1/person/123). Each additional route complicates caching; redirects are cheap for the server but slower for clients.

JSONLD does define a standard way to indicate that a value is a link: @id (which can be specified in a/an @context) https://www.w3.org/TR/json-ld11/

One additional downside to storing URIs instead of bare references is that it's more complicated to validate a URI template than a simple regex like \d+ or [abcdef\=\d+]+

[+]
[-]

No Python in Red Hat Linux 8?

/usr/bin/python can point to either /usr/bin/python3 or (as PEP 394 currently recommends) /usr/bin/python2

  $ alternatives --config python
FWIU, there are ubi8/python-27 and ubi8/python-36 docker images. IDK if they set /usr/bin/python out of the box? Changing existing shebangs may not be practical for some applications (which will need to specify 'python4' whenever that occurs over the next 10 supported years of RHEL/CENTOS 8)

[-]

JMAP: A modern, open email protocol

What are the optimizations in JMAP that make it faster than, say, Solid? Solid is built on a bunch of W3C Web, Security, and Linked Data Standards; LDP: Linked Data Protocol, JSON-LD: JSON Linked Data, WebID-TLS, REST, WebSockets, LDN: Linked Data Notifications. [1][2] Different worlds, I suppose.

There's no reason you couldn't represent RFC5322 data with RDF as JSONLD. There's now a way to do streaming JSON-LD.

LDP does paging and querying.

Solid supports pubsub with WebSockets and LDN. It may or may not (yet?) be as efficient for synchronization as JMAP, but it's definitely designed for all types of objects with linked data web standards; and client APIs can just parse JSON-LD.

[1] https://github.com/solid/information#solid-specifications

[2] https://github.com/solid/solid-spec/issues/123 "WebSockets and HTTP/2" SSE (Server-Side Events)

https://jmap.io/

JMAP: JSON Meta Application Protocol https://en.wikipedia.org/wiki/JSON_Meta_Application_Protocol

Is there a OpenAPI Specification for JMAP? There are a bunch of tools for Swagger / OpenAPIs: DRY interactive API docs, server implementations, code generators: https://swagger.io/tools/open-source/ https://openapi.tools/

Does JMAP support labels; such that I don't need to download a message and an attachment and mark it as read twice like labels over IMAP?

How does this integrate with webauthn; is that a different layer?

(edit) Other email things: openpgpjs; Web Key Directory /.well-known/openpgpkey/*; if there's no webserver on the MX domain, you can use the ACME DNS challenge to get free 3-month certs from LetsEncrypt.

https://wiki.gnupg.org/WKD

[+]

> If we hypothetically allow for equal adoption & mindshare of both, and assume both are non-terrible designs, I'd guess the one designed for "all types of objects" is less likely to ever be as efficient as the one designed with a single use-case in mind.

This is a generalization that is not supported by any data.

Standards enable competing solutions. Competing solutions often result in performance gains and efficiency.

Hopefully, there will be performant implementations and we won't need to reinvent the wheel in order to synchronize and send notifications for email, contacts, and calendars.

[+]

To eliminate the need for domain-specific parser implementations on both server and client, make it easy to index and search this structured data, and to link things with URIs and URLs like other web applications that also make lots of copies.

Solid is a platform for decentralized linked data storage and retrieval with access controls, notifications, WebID + OAuth/OpenID. The Wikipedia link and spec documents have a more complete description that could be retrieved and stored locally.

[-]

Grid Optimization Competition

From "California grid data is live – solar developers take note" https://news.ycombinator.com/item?id=18855820 :

>> It looks like California is at least two generations of technology ahead of other states. Let’s hope the rest of us catch up, so that we have a grid that can make an asset out of every building, every battery, and every solar system.

> +1. Are there any other states with similar grid data available for optimization; or any plans to require or voluntarily offer such a useful capability?

How do these competitions and the live actual data from California-only (so far; AFAIU) compare?

Are there standards for this grid data yet? Without standards, how generalizable are the competition solutions to real-world data?

[-]

Blockchain's present opportunity: data interchange standardization

What are the current standards efforts for blockchain data interchange?

W3C JSON-LD, ld-signatures + lds-merkleproof2017 (normalize the data before signing it so that the signature is representation-independent (JSONLD, RDFa, RDF, n-triples)), W3C DID Decentralized Identifiers, W3C Verifiable Claims, Blockcerts.org

W3C Credentials Community Group: https://w3c-ccg.github.io/community/work_items.html#draft-sp... (DID, Multihash (IETF), [...])

"Blockchain Credential Resources; a gist" https://gist.github.com/westurner/4345987bb29fca700f52163c33...

Specifically for payments:

https://www.w3.org/TR/?title=payment (the W3C Payment Request API standardizes browser UI payment/checkout workflows)

ILP: Interledger Protocol https://interledger.org/rfcs/0027-interledger-protocol-4/

> W3C JSON-LD

https://www.w3.org/TR/json-ld/ (JSON-LD 1.0)

https://www.w3.org/TR/json-ld11/ (JSON-LD 1.1)

> ld-signatures + lds-merkleproof2017 (normalize the data before signing it so that the signature is representation-independent (JSONLD, RDFa, RDF, n-triples))

https://w3c-dvcg.github.io/ld-signatures/

https://w3c-dvcg.github.io/lds-merkleproof2017/ (2017 Merkle Proof Linked Data Signature Suite)

> W3C DID Decentralized Identifiers

https://w3c-ccg.github.io/did-primer/

>> A Decentralized Identifier (DID) is a new type of identifier that is globally unique, resolveable with high availability, and cryptographically verifiable. DIDs are typically associated with cryptographic material, such as public keys, and service endpoints, for establishing secure communication channels. DIDs are useful for any application that benefits from self-administered, cryptographically verifiable identifiers such as personal identifiers, organizational identifiers, and identifiers for Internet of Things scenarios. For example, current commercial deployments of W3C Verifiable Credentials heavily utilize Decentralized Identifiers to identify people, organizations, and things and to achieve a number of security and privacy-protecting guarantees.

> W3C Verifiable Claims

https://github.com/w3c/verifiable-claims

https://w3c.github.io/vc-data-model/ (Data Model)

https://w3c.github.io/vc-use-cases/ (Use Cases: Education, Healthcare, Professional Credentials, Legal Identity,)

> Blockcerts.org

https://blockcerts.org/

[-]

Ask HN: Value of “Shares of Stock options” when joining a startup

I got an offer from a US start-up (well +25 employees) which has an office in EU where I would join them.

The offer's base salary is good (ie. higher than average for senior positions for that location) but I intend to negotiate it further, as I have possible other options. patio11's negotiation guide was a great read in that regard.

However, I'm relocating from a non-EU/US country, and I don't have a single idea about the financial systems, stock markets, and how to evaluate "15k shares of stock-options" or what "Stock Option and Grant Plan" means, I'm asking you fellow HNers about this part.

Do I just treat them as worthless and focus on base salary (as some internet sources suggest) or is there a formula to evaluate what they would be worth in say 2 years for instance ?

There are a number of options/equity calculators:

https://tldroptions.io/ ("~65% of companies will never exit", "~15% of companies will have low exits*", "~20% of companies will make you money")

https://comp.data.frontapp.com/ "Compensation and Equity Calculator"

http://optionsworth.com/ "What are my options worth?"

http://foundrs.com/ "Co-Founder Equity Calculator"

[-]

CMU Computer Systems: Self-Grading Lab Assignments (2018)

These look fun; in particular the "Attack Lab".

Dockerfiles might be helpful and easy to keep updated. Alpine Linux or just busybox are probably sufficient?

The instructor set could extend FROM the assignment image and run a few tests with e.g. testinfra (pytest)

You can also test code written in C with gtest.

I haven't read through all of the materials: are there suggested (automated) fuzzing tools? Does OSS-Fuzz solve?

Are there references to CWE and/or the SEI CERT C Coding Standard rules? https://wiki.sei.cmu.edu/confluence/plugins/servlet/mobile?c...

"How could we have changed our development process to catch these bugs/vulns before release?"

"If we have 100% [...] test coverage, would that mean we've prevented these vulns?"

What about 200%?

[+]
[+]

⟨100%| + |100%⟩ = 200%!

(Even code with 100% branch coverage may have common weaknesses like those that these (great) labs have students exploit)

[-]

Show HN: Debugging-Friendly Tracebacks for Python

cknd | 2019-04-28 14:50:29 | 121 | # | ^

pytest also has helpful tracebacks; though only for test runs.

With nose-progressive, you can specify --progressive-editor or update the .noserc so that traceback filepaths are prefixed with your preferred editor command.

vim-unstack parses paths from stack traces / tracebacks (for a number of languages including Python) and opens each in a split at that line number. https://github.com/mattboehm/vim-unstack

Here's the Python regex from my hackish pytb2paths.sh script:

  '\s+File "(?P<file>.*)", line (?P<lineno>\d+), in (?P<modulestr>.*)$'
https://github.com/westurner/dotfiles/blob/develop/scripts/p...

[-]

Why isn't 1 a prime number?

[+]
[+]

> You can also dial emergency contacts without unlocking the phone. They are accessible from the medical ID page on iOS, I assume Android has similar.

You can set a Lock Screen Message by searching for "Lock Screen Message" in the Android Settings.

You can also create an "ICE (In Case of Emergency)" contact.

[-]

Rare and strange ICD-10 codes

zdw | 2019-04-27 21:50:58 | 68 | # | ^
[+]
[+]
[+]

> No, you misunderstand the terminology. "Subsequent encounter" means with the doctor not with the rattlesnake

You can reference ICD codes with the schema.org/code property of schema.org/MedicalEntity and subclasses. https://schema.org/docs/meddocs.html

"Subsequent encounter" is poorly defined. IMHO, there should be a code for this.

[+]
[-]

Python Requests III

[+]

asyncio, HTTP/2, connection pooling, timeouts, Python 3.6+

README > "Feature Support" https://github.com/kennethreitz/requests3/blob/master/README...

[+]
[-]

Post-surgical deaths in Scotland drop by a third, attributed to a checklist

fanf2 | 2019-04-17 09:43:04 | 1036 | # | ^
[+]
[+]
[+]
[+]

GitHub and GitLab support task checklists in Markdown and also project boards which add and remove labels like 'ready' and 'in progress' when cards are moved between board columns; like kanban:

- [ ] not complete

- [x] completed

Other tools support additional per-task workflow states:

- [o] open

- [x (2019-04-17)] completed on date

I worked on a large hospital internal software project where the task was to build a system for reusable checklists editable through the web that prints them out in duplicate or triplicate at nearby printers. People really liked having the tangible paper copy.

"The Checklist Manifesto" by Atul Gawande was published while I worked there. TIL pilots have been using checklists for process control in order to reduce error for many years.

Evernote, RememberTheMilk, Google Tasks, and Google Keep all support checklists. Asana and Gitea and TaskWarrior support task dependencies.

A person could carry around a Hipster PDA with Bullet Journal style tasks lists with checkboxes; printed from a GTD service with an API and a @media print CSS stylesheet: https://en.wikipedia.org/wiki/Hipster_PDA

I'm not aware of very many tools that support authoring reusable checklists with structured data elements and data validation.

...

There are a number of configuration management systems like Puppet, Chef, Salt, and Ansible that build a graph of completable and verifiable tasks and then depth-first traverse said graph (either with hash randomization resulting in sometimes different traversals or with source order as an implicit ordering)

Resource scheduling systems like operating systems and conference room schedulers can take ~task priority into account when optimally ordering tasks given available resources; like triage.

Scheduling algorithms: https://news.ycombinator.com/item?id=15267146

TodoMVC catalogs Todo list implementations with very many MV* JS Frameworks: http://todomvc.com

[+]

For sure. Though many tools don't read .txt (or .md/.markdown) files.

GitHub and GitLab support (multiple) Issue and Pull Request templates:

Default: /.github/ISSUE_TEMPLATE.md || Configure in web interface

/.github/ISSUE_TEMPLATE/Name.md || /.gitlab/issue_templates/Name.md

Default: /.github/PULL_REQUEST_TEMPLATE.md || Configure in web interface

/.github/PULL_REQUEST_TEMPLATE/Name.md || /.gitlab/merge_request_templates/Name.md

There are template templates in awesome-github-templates [1] and checklist template templates in github-issue-templates [2].

[1] https://github.com/devspace/awesome-github-templates

[2] https://github.com/stevemao/github-issue-templates

[+]

Mattermost supports threaded replies and Markdown with checklist checkboxes

You can post GitHub/GitLab project updates to a Slack/Mattermost channel with webhooks (and search for and display GH/GL issues with /slash commands); though issue edits and checkbox state changes aren't (yet?) included in the events that channels receive.

[-]

Apply to Y Combinator

[+]
[+]

Here's the list of the 1,900 Y Combinator companies through Winter 2019 (W19) https://www.ycombinator.com/companies/

"Startup Playbook" by Sam Altman (YC Founder) and Illustrated by Gregory Koberger is also a good read: https://playbook.samaltman.com/

[-]

Trunk-Based Development vs. Git Flow

One major advantage of the gitflow/hubflow git workflows is that there is a standard way of merging across branches. For example, a 'hotfix' branch is merged into the stable master branch and also develop with one standard command; there's no need to re-explain and train new devs on how the branches were supposed to work here. I even copied the diagram(s) into my notes: https://westurner.github.io/tools/#hubflow

IMHO, `git log` on the stable master branch containing each and every tagged release is preferable to having multiple open release branches.

Requiring tests to pass before a PR gets merged is a good policy that's independent of the trunk or gitflow workflow decision.

[-]

Ask HN: Anyone else write the commit message before they start coding?

I feel like I just learned how to use Git: writing the message first thing has made me a lot more productive. I'm wondering if anyone else does this; I know test driven development is a thing, where people write tests before code, and this seems like a logical extension.

What a great idea. Are you updating the commit message with `git commit --amend` until you squash and push, or writing a novel on the side?

BDD acceptance tests can be written in a pseudo-prose syntax (and ideally, executed)

[-]

Ask HN: Datalog as the only language for web programming, logic and database

Can Datalog be used as the only language which we can use for writing server-side web application, complex domain business logic and database querying?

Are there any efforts made in this direction.

To quote myself from a post the other day https://news.ycombinator.com/item?id=19407170 :

> PyDatalog does Datalog (which is ~Prolog, but similar and very capable) logic programming with SQLAlchemy (and database indexes) and apparently NoSQL support. https://sites.google.com/site/pydatalog/

> Datalog: https://en.wikipedia.org/wiki/Datalog

> ... TBH, IDK about logic programming and bad facts. Resilience to incorrect and incredible information is - I suppose - a desirable feature of any learning system that reevaluates its learnings as additional and contradictory information makes its way into the datastores.

I'm not sure that Datalog is really necessary for most CRUD operations; SQLAlchemy and the SQLAlchemy ORM are generally sufficient for standard database querying CRUD.

[-]

Is there a program like codeacademy but for learning sysadmin?

if not, anyone wanna build one?

A few sysadmin and devops curriculum resources; though none but Beaker and Molecule are interactive with any sort of testing AFAIU:

"System Administrator" https://en.wikipedia.org/wiki/System_administrator

"Software Configuration Management" (SCM) https://en.wikipedia.org/wiki/Software_configuration_managem...

"DevOps" https://en.wikipedia.org/wiki/DevOps

"OpsSchool Curriculum" http://www.opsschool.org

- Soft Skills 101, 201

- Labs Exercises

- Free. Contribute

awesome-sysadmin > configuration-management https://github.com/kahun/awesome-sysadmin/blob/master/README...

- This could list reusable module collections such as Puppet Forge and Ansible Galaxy;

- And module testing tools like Puppet Beaker and Ansible Molecule (that can use Vagrant or Docker to test a [set of] machines)

https://github.com/stack72/ops-books

- I'd add "Time Management for System Administrators" (2005)

https://landing.google.com/sre/books/

- There's now a "Site Reliability Workbook" to go along with the Google SRE book. Both are free online.

https://response.pagerduty.com

- The PagerDuty Incident Response Documentation is also free online.

- OpsGenie has a free plan also with incident response alerting and on-call management.

There are a number of awesome-devops lists.

Minikube and microk8s package Kubernetes into a nice bundle of distributed systems components that'll run on Lin, Mac, Win. You can convert docker-compose.yml configs to Kubernetes pods when you decide that it should've been HA with a load balancer SPOF and x.509 certs and a DRP (Disaster Recovery Plan) from the start!

[-]

Maybe You Don't Need Kubernetes

ra7 | 2019-03-22 17:18:44 | 500 | # | ^
[+]
[+]

> As Kernighan said back in the 1970's, "Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?"

What a great quote. Thanks

[+]

> Tools like Ansible and Puppet, as great as they are, do not guarantee your infrastructure will end up in the state you defined and you easily end up with broken services.

False dilemma. Ansible and Puppet are great tools for configuring kubernetes, kubernetes worker nodes, and building container images.

Kubernetes does not solve for host OS maintenance; though there are a number of host OS projects which remove most of what they consider to be unnecessary services, there's still need to upgrade kubernetes nodes and move pods out of the way first (which can be done with e.g. Puppet or Ansible).

As well, it may not be appropriate for monitoring to depend upon kubernetes; there again you have nodes to manage with an SCM tool.

[-]

Quantum Machine Appears to Defy Universe’s Push for Disorder

[+]

"Scar (physics)": https://en.wikipedia.org/wiki/Scar_(physics)

> Scars are unexpected in the sense that stationary classical distributions at the same energy are completely uniform in space with no special concentrations along periodic orbits, and quantum chaos theory of energy spectra gave no hint of their existence

[-]

Pytype checks and infers types for your Python code

How does pytype compare with the PyAnnotate [1] and MonkeyType [2] dynamic / runtime PEP-484 type annotation type inference tools?

[1] https://github.com/dropbox/pyannotate

[2] https://github.com/Instagram/MonkeyType

[-]

How I'm able to take notes in mathematics lectures using LaTeX and Vim

[+]
[+]

> The mechanical task of taking notes is one of the most important parts of actually absorbing the material. It is not an either-or. Hearing/seeing the information, processing it in a way that makes sense to you individually, and then mechanically writing it down in a legible manner is one of the main methods that your brain learns. It's one of the primary reasons that taking notes is important in the first place. This is referred to as the "encoding hypothesis" [1].

There's almost certainly an advantage to learning to think about math using a publishable symbol set like LaTeX.

We learn by reinforcement; with feedback loops that may take until weeks later in a typical university course.

> There are actually even studies [2] that show that tools that assist in more efficient note taking, such as taking notes via typing rather than by hand, are actually detrimental to absorbing information, as it makes it easier for you to effectively pass the information directly from your ears to your computer without actually doing the processing that is required when writing notes by hand.

Handwriting notes is impractical for some people due to e.g. injury and illegibility.

The linked study regarding retention and handwritten versus typed notes has been debunked with references that are referenced elsewhere in comments on this post. There have been a few studies with insufficient controls (lack of randomization, for one) which have been widely repeated by educators who want to be given attention.

Doodling has been shown to increase information retention. Maybe doodling as a control really would be appropriate.

Banning laptops from lectures is not respectful of students with injury and illegible handwriting. Asking people to put their phones on silent (so they can still make and take emergency calls) and refrain from distracting other students with irrelevant content on their computers is reasonable and considerate.

(What a cool approach to math note-taking. I feel a bit inferior because I haven't committed to learning that valuable, helpful skill and so that's stupid and you're just wasting your time because that's not even necessary when all you need to do is retain the information you've paid for for the next few months at most. If course, once you get on the job, you'll never always be using that tool and e.g. latex2sympy to actually apply that theory to solving a problem that people are willing to pay for. So, thanks for the tips and kudos, idiot)

[-]

LHCb discovers matter-antimatter asymmetry in charm quarks

So, does this disprove all of supersymmetry? https://en.wikipedia.org/wiki/Supersymmetry

[+]

Ah, thanks.

"CPT Symmetry" https://en.wikipedia.org/wiki/CPT_symmetry

"CP Violations" https://en.wikipedia.org/wiki/CP_violation

"Charm quark" https://en.wikipedia.org/wiki/Charm_quark :

> The antiparticle of the charm quark is the charm antiquark (sometimes called anticharm quark or simply anticharm), which differs from it only in that some of its properties have equal magnitude but opposite sign.

[-]

React Router v5

[+]
[+]
[+]
[+]

Accidentally downvoted on mobile (and upvoted two others). Thanks for this.

"Scroll Restoration" https://reacttraining.com/react-router/web/guides/scroll-res...

[-]

Experimental rejection of observer-independence in the quantum world

Objective truth!? A question for epistemologists to decide.

How could they record their high entropy (?) solipsistic observations in an immutable datastore in such as way as to have probably zero knowledge of the other party's observations?

Anyways, that's why I only read the title and the abstract.

Wigner's friend experiment: https://en.wikipedia.org/wiki/Wigner%27s_friend

[-]

Show HN: A simple Prolog Interpreter written in a few lines of Python 3

Cool tests! PyDatalog does Datalog (which is ~Prolog, but similar and very capable) logic programming with SQLAlchemy (and database indexes) and apparently NoSQL support. https://sites.google.com/site/pydatalog/

Datalog: https://en.wikipedia.org/wiki/Datalog

... TBH, IDK about logic programming and bad facts. Resilience to incorrect and incredible information is - I suppose - a desirable feature of any learning system that reevaluates its learnings as additional and contradictory information makes its way into the datastores.

[+]
[-]

How to earn your macroeconomics and finance white belt as a software developer

Thanks for the wealth of resources in this post. Here are a few more:

"Python for Finance: Analyze Big Financial Data" (2014, 2018) https://g.co/kgs/qkY8J6 ... https://pyalgo.tpq.io also includes the "Finance with Python" course and this book as a PDF and Jupyter notebooks.

Quantopian put out a call for the best Value Investing algos (implemented in quantopian/zipline) awhile back. This post links to those and other value investing resources: https://westurner.github.io/hnlog/#comment-19181453 (Ctrl-F "econo")

"Lectures in Quantitative Economics as Python and Julia Notebooks" https://news.ycombinator.com/item?id=19083479 links to these excellent lectures and a number of tools for working with actual data from FRED, ECB, Eurostat, ILO, IMF, OECD, UNSD, UNESCO, World Bank, Quandl.

One thing that many finance majors, courses, and resources often fail to identify is the role that startup and small businesses play in economic growth and actual value creation: jobs, GDP, return on direct capital investment. Most do not succeed, but it is possible to do better than index funds and have far more impact in terms of sustainable investment than as an owner of a nearly-sure-bet index fund that owns some shares and takes a hands-off approach to business management, research, product development, and operations.

Is it possible to possess a comprehensive understanding of finance and economics but still not have personal finance down? Personal finance: r/personalfinance/wiki, "Consumer science (a.k.a. home economics) as a college major" https://news.ycombinator.com/item?id=17894632

[-]

Ask HN: Relationship between set theory and category theory

I have an idea about the relationship between set theory and category theory and I would like some feedback. I would like others to see it too, and I don't know how to do it. I think it's at least interesting to look at as a slightly crazy collage, but I was a bit more excited than normal when the idea hit, so I just had to dump it all at once in this image: https://twitter.com/FamilialRhino/status/1101777965724168193 (You will have to zoom the picture in order to be able to read the scribbles.)

It has to do with resonance in the energy flowing in emergent networks. Can't quite put my finger on it, so I'll be here to answer any questions.

Thanks for reading.

"Categorical set theory" > "References" https://en.wikipedia.org/wiki/Categorical_set_theory#Referen...

From "Homotopy category" > "Concrete categories" https://en.wikipedia.org/wiki/Homotopy_category#Concrete_cat... :

> While the objects of a homotopy category are sets (with additional structure), the morphisms are not actual functions between them, but rather a classes of functions (in the naive homotopy category) or "zigzags" of functions (in the homotopy category). Indeed, Freyd showed that neither the naive homotopy category of pointed spaces nor the homotopy category of pointed spaces is a concrete category. That is, there is no faithful functor from these categories to the category of sets.

[+]
[-]

The most popular docker images each contain at least 30 vulnerabilities

[+]
[+]
[+]
[+]
[+]

I don't think this is a tooling problem at all.

"The tooling makes it too easy to do it wrong." Compared to shell scripts with package manager invocations? Nobody configures a system with just packages: there are always scripts to call, chroots to create, users and groups to create, passwords to set, firewall policies to update, etc.

There are a bunch of ways to create LXC containers: shell scripts, Docker, ansible. Shell scripts preceded Docker: you can write a function to stop, create an intermediate tarball, and then proceed (so that you don't have to run e.g. debootstrap without a mirror every time you manually test your system build script; so that you can cache build steps that completed successfully).

With Docker images, the correct thing to do is to extend FROM the image you want to use, build the whole thing yourself, and then tag and store your image in a container repository. Neither should you rely upon months-old liveCD images.

"You should just build containers on busybox." So, no package management? A whole ensemble of custom builds to manually maintain (with no AppArmor or SELinux labels)? Maintainers may prefer for distros to field bug reports for their own common build configurations and known-good package sets. Please don't run as root in a container ("because it's only a container that'll get restarted someday"). Busybox is not a sufficient OS distribution.

It's not the tools, it's how people are choosing to use them. They can, could, and should try and use idempotent package management tasks within their container build scripts; but they don't and that's not Bash/Ash/POSIX's fault either.

> With Docker images, the correct thing to do is to extend FROM the image you want to use, build the whole thing yourself, and then tag and store your image in a container repository. Neither should you rely upon months-old liveCD images.

This should rebuild all. There should be an e.g. `apt-get upgrade -y && rm -rf /var/lib/apt/lists` in there somewhere (because base images are usually not totally current (and neither are install ISOs)).

`docker build --no-cache --pull`

You should check that each Dockerfile extends FROM `tag:latest` or the latest version of the tag that you support. Its' not magical, you do have to work it.

Also, IMHO, Docker SHOULD NOT create another Linux distribution.

[-]

Tinycoin: A small, horrible cryptocurrency in Python for educational purposes

The 'dumbcoin' jupyter notebook is also a good reference: "Dumbcoin - An educational python implementation of a bitcoin-like blockchain" https://nbviewer.jupyter.org/github/julienr/ipynb_playground...

[-]

When does the concept of equilibrium work in economics?

"Modeling stock return distributions with a quantum harmonic oscillator" (2018) https://iopscience.iop.org/article/10.1209/0295-5075/120/380...

> We propose a quantum harmonic oscillator as a model for the market force which draws a stock return from short-run fluctuations to the long-run equilibrium. The stochastic equation governing our model is transformed into a Schrödinger equation, the solution of which features "quantized" eigenfunctions. Consequently, stock returns follow a mixed χ distribution, which describes Gaussian and non-Gaussian features. Analyzing the Financial Times Stock Exchange (FTSE) All Share Index, we demonstrate that our model outperforms traditional stochastic process models, e.g., the geometric Brownian motion and the Heston model, with smaller fitting errors and better goodness-of-fit statistics. In addition, making use of analogy, we provide an economic rationale of the physics concepts such as the eigenstate, eigenenergy, and angular frequency, which sheds light on the relationship between finance and econophysics literature.

"Quantum harmonic oscillator" https://en.wikipedia.org/wiki/Quantum_harmonic_oscillator

The QuantEcon lectures have a few different multiple agent models:

"Rational Expectations Equilibrium" https://lectures.quantecon.org/py/rational_expectations.html

"Markov Perfect Equilibrium" https://lectures.quantecon.org/py/markov_perf.html

"Robust Markov Perfect Equilibrium" https://lectures.quantecon.org/py/rob_markov_perf.html

"Competitive Equilibria of Chang Model" https://lectures.quantecon.org/py/chang_ramsey.html

... "Lectures in Quantitative Economics as Python and Julia Notebooks" https://news.ycombinator.com/item?id=19083479 (data sources (pandas-datareader, pandaSDMX), tools, latex2sympy)

"Econophysics" https://en.wikipedia.org/wiki/Econophysics

> Indeed, as shown by Bruna Ingrao and Giorgio Israel, general equilibrium theory in economics is based on the physical concept of mechanical equilibrium.

[-]

Simdjson – Parsing Gigabytes of JSON per Second

> Requirements: […] A processor with AVX2 (i.e., Intel processors starting with the Haswell microarchitecture released 2013, and processors from AMD starting with the Rizen)

[+]
[+]
[+]
[+]
[+]
[+]
[-]

A faster, more efficient cryptocurrency

[+]

Are there reasons that e.g. Bitcoin and Ethereum and Stellar could not implement some of these more performant approaches that Algorand [1] and Vault [2] have developed, published, and implemented? Which would require a hard fork?

[1] https://www.algorand.com/

[2] https://dspace.mit.edu/handle/1721.1/117821

[+]

And what of decentralized premined chains (with no PoW, no PoS, and far less energy use) that release coins with escrow smart contracts over time such as Ripple and Stellar (and close a new ledger every few seconds)?

> Algorand has a very fast consensus mechanism and can add blocks as quickly as the network can deliver them. We become a victim of our success. The blockchain will grow very rapidly. A terabyte a month is possible. The storage issue associated with our performance can quickly become an issue. The Vault paper is focused on solving this and other storage scaling problems.

What prevents a person from using a chain like IPFS?

Ethereum Casper PoS has been under review for quite some time.

Why isn't all Bitcoin on Lightning Network?

Bitcoin could make bootstrapping faster by choosing a considered-good blockhash and balances, but AFAIU, re-verifying transactions like Bitcoin and derivatives do prevents hash collision attacks that are currently considered infeasible for SHA-256 (especially given a low block size).

There was an analysis somewhere where they calculated the cloud server instance costs of mounting a ~51% attack (which applies to PoW chains) for various blockchains.

Bitcoin is not profitable to mine in places without heavily subsidized dirty/clean energy anymore: energy and Bitcoin commodity costs and prices have intersected. They'll need any of: inexpensive clean energy, more efficient chips, higher speculative value.

Energy arbitrage (grid-scale energy storage) may be more profitable now. We need energy storage in order to reach 100% renewable energy (regardless of floundering policy support).

[+]

People argue this all day. There's a lot of FUD.

Ripple only runs ~7% of validator nodes; which is far less centralized control than major Bitcoin mining pools and businesses (who do the deciding in regards to the many Bitcoin hard forks); that's one form of decentralization.

Ripple clients can use their own UNL or use the Ripple-approved UNL.

Ripple is traded on a number of exchanges (though fewer than Bitcoin for certain); that's another form of decentralization.

As an open standard, ILP will further reduce vendor lock in (and increase interoperability between) networks that choose to implement it.

There are forks of Ripple (e.g. Stellar) just like there are forks of Bitcoin and Ethereum.

From https://ripple.com/insights/the-inherently-decentralized-nat... :

> In contrast, the XRP Ledger requires 80 percent of validators on the entire network, over a two-week period, to continuously support a change before it is applied. Of the approximately 150 validators today, Ripple runs only 10. Unlike Bitcoin and Ethereum — where one miner could have 51 percent of the hashing power — each Ripple validator only has one vote in support of an exchange or ordering a transaction.

How does your definition of 'decentralized' differ?

[+]
[-]

Git-signatures – Multiple PGP signatures for your commits

[+]
[+]

> I think it is probably in the class of problems where there are no great foolproof solutions. However, I can imagine that techniques like certificate transparency (all signed x509 certificates pushed to a shared log) would be quite useful.

Securing DNS: "https://news.ycombinator.com/item?id=19181362"

> Certs on the Blockchain: "Can we merge Certificate Transparency with blockchain?" https://news.ycombinator.com/item?id=18961724

> Namecoin (decentralized blockchain DNS): https://en.wikipedia.org/wiki/Namecoin

[+]

My mistake. How ironic. Everything depends upon the red wheelbarrow. Here's that link without the trailing ": https://news.ycombinator.com/item?id=19181362

> My main problem with blockchain is the excessive energy consumption of PoW. I know there are PoS efforts, but they seem problematical.

One report said that 78% of Bitcoin energy usage is from renewable sources (many of which would otherwise be curtailed and otherwise unfunded due to flat-to-falling demand for electricity). But PoW really is expensive and hopefully the market will choose less energy-inefficient solutions from the existing and future blockchain solutions while keeping equal or better security assurances.

>> Proof of Work (Bitcoin, ...), Proof of Stake (Ethereum Casper), Proof of Space, Proof of Research (GridCoin, CureCoin,)

The spec should be: DDOS resiliant (without a SPOF), no one entity with control over API and/or database credentials and database backups and the clock, and immutable.

Immutability really cannot be ensured with hashed records that incorporate the previous record's hash as a salt in a blocking centralized database because someone ultimately has root and the clock and all the backups and code vulnerable to e.g. [No]SQL injection; though distributed 'replication' and detection of record modification could be implemented. git push -f may be detected if it's on an already-replicated branch; but git depends upon local timestamps. google/trillian does Merkle trees in a centralized database (for Certificate Transparency).

In quickly reading the git-signatures shell script sources, I wasn't certain whether the git-notes branch with the .gitsigners that are fetched from all n keyservers (with DNS) is also signed?

I also like the "Table 1: Security comparison of Log Based Approaches to Certificate Management" in the CertLedger paper. Others are far more qualified to compare implementations.

[+]

> I'd love if it could be rooted in a Yubikey.

FIDO2 and Yubico helped develop the new W3C WebAuthn standard: https://en.wikipedia.org/wiki/WebAuthn

But WebAuthn does not solve for WoT or PKI or certificate pinning.

> Decoupling the "signing" and "verifying" parts seem like a good idea. As random Person signs something, how someone else figures out how to go trust that signature is a separate problem.

Someone can probably help with terminology here. There's identification (proving that a person has the key AND that it's their key (biometrics, challenge-response)), signing (using a key to create a cryptographic signature – for the actual data or a reasonably secure cryptographic hash of said data – that could only could have been created with the given key), signature verification (checking that the signature was created by the claimed key for the given data), and then there's trusting that the given key is authorized for a specific purpose (Web of Trust (key-signing parties), PKI, ACME, exchange of symmetric keys over a different channel such as QKD) by e.g. signing a structured document that links cryptographic keys with keys for specific authorized functions and trusting the key(s) used to sign said authorizing document.

Private (e.g. Zero Knowledge) blockchains can be used for key exchange and key rotation. Public blockchains can be used for sharing (high-entropy) key components; also with an optional exchange of money to increase the cost of key compromise attempts.

There's also WKD: "Web Key Directory"; which hosts GPG keys over HTTPS from a .well-known URL for a given user@domain identifier: https://wiki.gnupg.org/WKD

Compared to existing PGP/GPG keyservers, WKD does rely upon HTTPS.

TUF is based on Thandy. TUF: "The Update Framework" does not presume channel security (is designed to withstand channel compromise) https://en.wikipedia.org/wiki/The_Update_Framework_(TUF)

The TUF spec doesn't mention PGP/GPG: https://github.com/theupdateframework/specification/blob/mas...

There's a derivative of TUF for automotive applications called Uptane: https://uptane.github.io

The Bitcoin article on multisignature; 1-of-2, 2-of-2, 2-of-3, 3-of-5, etc.: https://en.bitcoin.it/wiki/Multisignature

[-]

Compounding Knowledge

[+]
[+]
[+]
[+]

BTW, AQR funded the initial development of pandas; which now powers tools like alphalens (predictive factor analysis) and pyfolio.

There's your 'compounding knowledge'.

(Days later)

"7 Best Community-Built Value Investing Algorithms Using Fundamentals" https://blog.quantopian.com/fundamentals-contest-winners/

(The Zipline backtesting library also builds upon Pandas)

How can we factor ESG/sustainability reporting into these fundamentals-driven algorithms in order to save the world?

[+]
[+]

"The Superinvestors of Graham and Doddsville" (1984) https://scholar.google.com/scholar?cluster=17265410477248371...

From https://en.wikipedia.org/wiki/The_Superinvestors_of_Graham-a... :

> The speech and article challenged the idea that equity markets are efficient through a study of nine successful investment funds generating long-term returns above the market index.

This book probably doesn't mention that he's given away over 71% to charity since Y2K. Or that it's really cold and windy and snowy in Omaha; which makes for lots of reading time.

"Warren Buffett and the Interpretation of Financial Statements: The Search for the Company with a Durable Competitive Advantage" (2008) [1], "Buffetology" (1999) [2], and "The Intelligent Investor" (1949, 2009) [3] are more investment-strategy-focused texts.

[1] https://smile.amazon.com/Warren-Buffett-Interpretation-Finan...

[2] https://smile.amazon.com/Buffettology-Previously-Unexplained...

[3] https://smile.amazon.com/Intelligent-Investor-Definitive-Inv...

Value Investing: https://en.wikipedia.org/wiki/Value_investing https://www.investopedia.com/terms/v/valueinvesting.asp

> This is why it’s commonly telling you what happened, not why it happened or under what conditions it might happen again.

[-]

Why CISA Issued Our First Emergency Directive

There are a number of efforts to secure DNS (and SSL/TLS which generally depends upon DNS; and upon which DNS-over-HTTPS depends) and the identity proof systems which are used for record-change authentication and authorization.

Domain registrars can and SHOULD implement multi-factor authentication. https://en.wikipedia.org/wiki/Multi-factor_authentication

Are there domain registrars that support FIDO/U2F or the new W3C WebAuthn spec? https://en.wikipedia.org/wiki/WebAuthn

Credentials and blockchains (and biometrics): https://gist.github.com/westurner/4345987bb29fca700f52163c33...

DNSSEC: https://en.wikipedia.org/wiki/Domain_Name_System_Security_Ex...

ACME / LetsEncrypt certs expire after 3 months (*) and require various proofs of domain ownership: https://en.wikipedia.org/wiki/Automated_Certificate_Manageme...

Certificate Transparency: https://en.wikipedia.org/wiki/Certificate_Transparency

Certs on the Blockchain: "Can we merge Certificate Transparency with blockchain?" https://news.ycombinator.com/item?id=18961724

Namecoin (decentralized blockchain DNS): https://en.wikipedia.org/wiki/Namecoin

DNSCrypt: https://en.wikipedia.org/wiki/DNSCrypt

DNS over HTTPS: https://en.wikipedia.org/wiki/DNS_over_HTTPS

DNS over TLS: https://en.wikipedia.org/wiki/DNS_over_TLS

DNS: https://en.wikipedia.org/wiki/Domain_Name_System

[-]

Chrome will Soon Let You Share Links to a Specific Word or Sentence on a Page

[+]

"Integration with W3C Web Annotations" https://github.com/bokand/ScrollToTextFragment/issues/4

> It would be great to be able to comment on the linked resource text fragment. W3C Web Annotations [implementations] don't recognize the targetText parameter, so AFAIU comments are then added to the document#fragment and not the specified text fragment. [...]

> Is there a simplified mapping of W3C Web Annotations to URI fragment parameters?

[-]

Guidelines for keeping a laboratory notebook

[+]

> Computation related fields lend themselves well to purely electronic notebooks, no surprise. Today, a lot of my work fits perfectly in a Jupyter notebook.

Some notes and ideas regarding Jupyter notebooks as lab notebooks from "Keeping a Lab Notebook [pdf]": https://news.ycombinator.com/item?id=15710815

[-]

Superalgos and the Trading Singularity

Though others didn't, you might find this interesting: "Ask HN: Why would anyone share trading algorithms and compare by performance?" https://news.ycombinator.com/item?id=15802785 ( https://westurner.github.io/hnlog/#story-15802785 )

[+]

I think part of the value of sharing knowledge and algorithmic implementations comes from getting feedback from other experts; like peer review and open science and teaching.

Case in point: the first algorithm on this list [1] of community contributed algorithms that were migrated to their new platform is "minimum variance w/ constraint" [2]. Said algorithm showed returns of over 200% as compared with 77% returns from the SPY S&P 500 ETF over the same period, ceteris paribus. In the 69 replies, there are modifications by community members and the original author that exceed 300%.

Working together on open algorithms has positive returns that may exceed advantages of closed algorithmic development without peer review.

[1] https://www.quantopian.com/posts/community-algorithms-migrat...

[2] https://www.quantopian.com/posts/56b6021b3f3b36b519000924

[+]

> How well does it do in production though and what happens when multiple algos execute the same trades?

Price inflation.

> Does it cause the rest of the algos to adapt and change results?

Trading index ETFs? IDK

> It makes sense to back-test together and work on it, but if it's proven to work, someone will create something to monitor volume on those trades and work against it.

Why does it need to do lots of trades? Is it possible for anyone other than e.g. SEC to review trades by buyer or seller?

> I'd be curious to see the same algo do 300% in production, and if so, then my bias would be uncalled for.

pyfolio does tear sheets with Zipline algos: pyfolio/examples/zipline_algo_example.ipynb https://nbviewer.jupyter.org/github/quantopian/pyfolio/blob/...

alphalens does performance analysis of predictive factors: alphalens/examples/pyfolio_integration.ipynb https://nbviewer.jupyter.org/github/quantopian/alphalens/blo...

awesome-quant lists a bunch of other tools for algos and superalgos: https://github.com/wilsonfreitas/awesome-quant

What's a good platform for paper trading (with e.g. zipline or moonshot algorithms)?

[+]
[-]

Crunching 200 years of stock, bond, currency and commodity data

[+]
[+]
[+]

I was interested, so I did some research here.

Rational Choice Theory https://en.wikipedia.org/wiki/Rational_choice_theory

Rational Behavior https://www.investopedia.com/terms/r/rational-behavior.asp

> Most mainstream academic economics theories are based on rational choice theory.

> While most conventional economic theories assume rational behavior on the part of consumers and investors, behavioral finance is a field of study that substitutes the idea of “normal” people for perfectly rational ones. It allows for issues of psychology and emotion to enter the equation, understanding that these factors alter the actions of investors, and can lead to decisions that may not appear to be entirely rational or logical in nature. This can include making decisions based primarily on emotion, such as investing in a company for which the investor has positive feelings, even if financial models suggest the investment is not wise.

Behavioral finance https://www.investopedia.com/terms/b/behavioralfinance.asp

Bounded rationality > Relationship to behavioral economics https://en.wikipedia.org/wiki/Bounded_rationality

Perfectly rational decisions can be and are made without perfect information; bounded by the information available at the time. If we all had perfect information, there would be no entropy and no advantage; just lag and delay between credible reports and order entry.

Information asymmetry https://en.wikipedia.org/wiki/Information_asymmetry

Heed these words wisely: What foolish games! Always breaking my heart.

https://deepmind.com/blog/game-theory-insights-asymmetric-mu...

> Asymmetric games also naturally model certain real-world scenarios such as automated auctions where buyers and sellers operate with different motivations. Our results give us new insights into these situations and reveal a surprisingly simple way to analyse them. While our interest is in how this theory applies to the interaction of multiple AI systems, we believe the results could also be of use in economics, evolutionary biology and empirical game theory among others.

https://en.wikipedia.org/wiki/Pareto_efficiency

> A Pareto improvement is a change to a different allocation that makes at least one individual or preference criterion better off without making any other individual or preference criterion worse off, given a certain initial allocation of goods among a set of individuals. An allocation is defined as "Pareto efficient" or "Pareto optimal" when no further Pareto improvements can be made, in which case we are assumed to have reached Pareto optimality.

Which, I think, brings me to equitable availability of maximum superalgo efficiency and limits of real value creation in capital and commodities markets; which'll have to be a topic for a different day.

[-]

Show HN: React-Schemaorg: Strongly-Typed Schema.org JSON-LD for React

[+]

https://dev.to/eyassh/react-schemaorg-strongly-typed-schemao...

Is there a good way to generate JSONschema and thus forms from schema.org RDFS classes and (nested, repeatable) properties?

[+]

There are a number of tools for generating forms and requisite client and serverside data validations from JSONschema; but I'm not aware of any for RDFS (and thus the schema.org schema [1]). A different use case, for certain.

https://schema.org/docs/developers.html#defs

[+]
[-]

Consumer Protection Bureau Aims to Roll Back Rules for Payday Lending

From the article:

> The way payday loans work is that payday lenders typically offer small loans to borrowers who promise to pay the loans back by their next paycheck. Interest on the loans can have an annual percentage rate of 390 percent or more, according to a 2013 report by the CFPB. Another bureau report from the following year found that most payday loans — as many as 80 percent — are rolled over into another loan within two weeks. Borrowers often take out eight or more loans a year.

390%

From https://www.npr.org/2019/02/06/691944789/consumer-protection... :

> TARP recovered funds totalling $441.7 billion from $426.4 billion invested, earning a $15.3 billion profit or an annualized rate of return of 0.6% and perhaps a loss when adjusted for inflation.[2][3]

0.6%

[+]
[+]
[+]

Lectures in Quantitative Economics as Python and Julia Notebooks

[+]
[+]
[+]
[+]

You can build something like this with Jupyter today.

> Traitlets is a framework that lets Python classes have attributes with type checking, dynamically calculated default values, and ‘on change’ callbacks. https://traitlets.readthedocs.io/en/stable/

> Traitlet events. Widget properties are IPython traitlets and traitlets are eventful. To handle changes, the observe method of the widget can be used to register a callback https://ipywidgets.readthedocs.io/en/stable/examples/Widget%...

You can definitely build interactive notebooks with Jupyter Notebook and JupyterLab (and ipywidgets or Altair or HoloViews and Bokeh or Plotly for interactive data visualization).

> Qgrid is a Jupyter notebook widget which uses SlickGrid to render pandas DataFrames within a Jupyter notebook. This allows you to explore your DataFrames with intuitive scrolling, sorting, and filtering controls, as well as edit your DataFrames by double clicking cells. https://github.com/quantopian/qgrid

Qgrid's API includes event handler registration: https://qgrid.readthedocs.io/en/latest/

> neuron is a robust application that seamlessly combines the power of Visual Studio Code with the interactivity of Jupyter Notebook. https://marketplace.visualstudio.com/items?itemName=neuron.n...

"Excel team considering Python as scripting language: asking for feedback" (2017) https://news.ycombinator.com/item?id=15927132

OpenOffice Calc ships with Python 2.7 support: https://wiki.openoffice.org/wiki/Python

Procedural scripts written in a general purpose language with named variables (with no UI input except for chart design and persisted parameter changes) are reproducible.

What's a good way to review all of the formulas and VBA and/or Python and data ETL in a spreadsheet?

Is there a way to record a reproducible data transformation script from a sequence of GUI interactions in e.g. OpenRefine or similar?

OpenRefine/OpenRefine/wiki/Jupyter

"Within the Python context, a Python OpenRefine client allows a user to script interactions within a Jupyter notebook against an OpenRefine application instance, essentially as a headless service (although workflows are possible where both notebook-scripted and live interactions take place. https://github.com/OpenRefine/OpenRefine/wiki/Jupyter

Are there data wrangling workflows that are supported by OpenRefine but not Pandas, Dask, or Vaex?

[+]

There are undergraduate and graduate courses in each language:

Python version: https://lectures.quantecon.org/py/

Julia version: https://lectures.quantecon.org/jl/

[+]

pandas-datareader can pull data from e.g. FRED, Eurostat, Quandl, World Bank: https://pandas-datareader.readthedocs.io/en/latest/remote_da...

pandaSDMX can pull SDMX data from e.g. ECB, Eurostat, ILO, IMF, OECD, UNSD, UNESCO, World Bank; with requests-cache for caching data requests: https://pandasdmx.readthedocs.io/en/latest/#supported-data-p...

The scikit-learn estimator interface includes a .score() method. "3.3. Model evaluation: quantifying the quality of predictions" https://scikit-learn.org/stable/modules/model_evaluation.htm...

statsmodels also has various functions for statistically testing models: https://www.statsmodels.org/stable/

"latex2sympy parses LaTeX math expressions and converts it into the equivalent SymPy form" and is now merged into SymPy master and callable with sympy.parsing.latex.parse_latex(). It requires antlr-python-runtime to be installed. https://github.com/augustt198/latex2sympy https://github.com/sympy/sympy/pull/13706

IDK what Julia has for economic data retrieval and model scoring / cost functions?

[-]

If Software Is Funded from a Public Source, Its Code Should Be Open Source

From the US Digital Services Playbook [1]:

> PLAY 13

> Default to open

> When we collaborate in the open and publish our data publicly, we can improve Government together. By building services more openly and publishing open data, we simplify the public’s access to government services and information, allow the public to contribute easily, and enable reuse by entrepreneurs, nonprofits, other agencies, and the public.

> Checklist

> - Offer users a mechanism to report bugs and issues, and be responsive to these reports

> [...]

> - Ensure that we maintain contractual rights to all custom software developed by third parties in a manner that is publishable and reusable at no cost

> [...]

> - When appropriate, publish source code of projects or components online

> [...]

> Key Questions

> [...]

> - If the codebase has not been released under an open source license, explain why.

> - What components are made available to the public as open source?

> [...]

[1] https://playbook.cio.gov/#play13

Apache Arrow 0.12.0

> Apache Arrow is a cross-language development platform for in-memory data. It specifies a standardized language-independent columnar memory format for flat and hierarchical data, organized for efficient analytic operations on modern hardware. It also provides computational libraries and zero-copy streaming messaging and interprocess communication. Languages currently supported include C, C++, C#, Go, Java, JavaScript, MATLAB, Python, R, Ruby, and Rust.

Statement on Status of the Consolidated Audit Trail (2018)

> Put simply, the CAT is intended to enable regulators to oversee the securities markets on a consolidated basis—and in so doing, better protect these markets and investors.

[-]

Post Quantum Crypto Standardization Process – Second Round Candidates Announced

> As the latest step in its program to develop effective defenses, the National Institute of Standards and Technology (NIST) has winnowed the group of potential encryption tools—known as cryptographic algorithms—down to a bracket of 26. These algorithms are the ones NIST mathematicians and computer scientists consider to be the strongest candidates submitted to its Post-Quantum Cryptography Standardization project, whose goal is to create a set of standards for protecting electronic information from attack by the computers of both tomorrow and today.

> “These 26 algorithms are the ones we are considering for potential standardization, and for the next 12 months we are requesting that the cryptography community focus on analyzing their performance,”

Links to the 17 public-key encryption and key-establishment algorithms and 9 digital signature algorithms are here: "Round 2 Submissions" https://csrc.nist.gov/Projects/Post-Quantum-Cryptography/Rou...

"Quantum Algorithm Zoo" has moved to https://quantumalgorithmzoo.org .

[-]

Ask HN: How do you evaluate security of OSS before importing?

What tools can I use to evaluate the security posture of an OSS project before I approve its usage with high confidence?

Oddly, whether a project has at least one CVE reported could be interpreted in favor of the project. https://www.cvedetails.com

Do they have a security disclosure policy? A dedicated security mailing list?

Do they pay bounties or participate in e.g Pwn2own?

Do they cryptographically sign releases?

Do they cryptographically sign VCS tags (~releases)? commits? `git tag -s` / `git commit/merge -S` https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work

Downstream packagers do sometimes/often apply additional patches and then sign their release with the repo (and thus system global) GPG key.

Whether they require "Signed-off-by" may indicate that the project has mature controls and possibly a formal code review process requirement. (Look for "Signed-off-by:" in the release branch (`git commit/merge -s/--signoff`)

How have they integrated security review into their [iterative] release workflow?

Is the software formally verified? Are parts of the software implementation or spec formally verified?

Does the system trust the channel? The host? Is it a 'trustless' system?

What are the single points of failure?

How is logging configured? To syslog?

Do they run the app as root in a Docker container? Does it require privileged containers?

If it has to run as root, does it drop privileges at startup?

Does the package have an SELinux or AppArmor policy? (Or does it say e.g. "just set SELinux to permissive mode)

Is there someone you can pay to support the software in an enterprise environment? Open or closed, such contacts basically never accept liability; but if there is an SLA, do you get a pro-rated bill?

As far as indicators of actual software quality:

How much test coverage is there? Line coverage or statement coverage?

Do they run static analysis tools for all pull requests and releases? Dynamic analysis? Fuzzing?

Of course, closed or open source projects may do none or all of these and still be totally secure, insecure, or unsecure.

[+]
[-]

Ask HN: How can I use my programming skills to support nonprofit organizations?

Lately I've been thinking about doing programming for nonprofits, both because I want to help out with what I'm good at but also to hone my skills and potentially get some open source credit.

So far I've had a hard time finding nonprofit projects where I can just pick up something and start programming. I know about freecodecamp.org, but they force you to go through their courses, and as I already have multiple years of experience as a developer, I feel like that would be a waste of time.

Isn't there a way to contribute to nonprofit organization in a more direct and simple manner like how you would contribute to an open source project on GitHub?

There are lots of project management systems with issue tracking and kanban boards with swimlanes. Because it's unreasonable to expect all volunteers to have a GH account or even understand what GH is for, support for external identity management and SSO may be essential to getting people to actually log in and change their password regularly.

Sidling a nonprofit with custom built software with no other maintainers is not what they need. Build (and pay for development, maintenance, timely security upgrades and security review) or Buy (where is our data? who backs it up? how much does it cost for a month or a few years? Is it open source with a hosted option; so that we can pay a developer to add or fix what we need?)

"Solutions architect" may be a more helpful objective title for what's needed. https://en.wikipedia.org/wiki/Solution_architecture

What are their needs? Marketing, accounting, operations, HR

Marketing: web site, maps service, directions, active social media presence that speaks to their defined audience

Accounting: Revenue and expenses, payroll/benefits/HR, projections, "How can we afford to do more?", handle donations and send receipts for tax purposes, reports to e.g. https://charitynavigator.org/ and infographics for wealth-savvy donors

Operations: Asset inventory, project management, volunteer scheduling

HR: payroll, benefits, volunteer scheduling, training, turnover, retaining selfless and enlightenedly-self-interested volunteers

Create a spreadsheet. Rows: needs/features/business processes. Columns: essential, nice to have, software products and services.

Create another spreadsheet. Rows: APIs. Columns: APIs.

Training: what are the [information systems] processes/workflows/checklists? How can I suggest a change? How do we reach consensus that there's a better way to do this? Is there a wiki? Is there a Q&A system?

"How much did you sink on that? Probably seemed like the best option according to the information available at the time, huh? Do you have a formal systems acquisition process? Who votes according to what type of prepared analysis? How much would it cost to switch? What do we need to do to ETL (extract, transform, and load) into a newer better system?"

When estimating TCO for a nonprofit, turnover is a very real consideration. People move. Chances are, as with most organizations TBH, there's a patchwork of partially-integrated and maybe-integrable systems that it may or may not be more cost-effective and maintainable to replace with a cloud ERP specifically designed for nonprofits.

Who has access rights to manually update which parts of the website? How can we include dynamic ([other] database-backed) content in our website? What is a CMS? What is an ERP? What is a CRM? Are these customers, constituents, or both? When did we last speak with those guys? How can people share our asks with social media networks?

If you're not willing or able to make a long-term commitment, the more responsible thing to do is probably to disclose any conflicts of interest recommend a SaaS solution hosted in a compliant data center.

q="nonprofit erp"

q="nonprofit crm"

q="nonprofit cms" + donation campaign visibility

What time of day are social media posts most likely to get maximum engagement from which segments of our audience? What is our ~ARPU "average revenue per user/follower"?

... As a volunteer and not a FTE, it may be a worthwhile exercise to build a prototype of the new functionality with whatever tools you happen to be familiar with with the expectation that they'll figure out a way to accomplish the same objectives with their existing systems. If that's not possible, there may be a business opportunity: are there other organizations with the same need? Is there a sustainable market for such a solution? You may be building to be acquired.

[-]

Ask HN: Steps to forming a company?

Hey guys, I'm leaving my firm very shortly to form a startup.

Does why have a checklist of proper ways to do things?

Ie. 1. Form Chapter C Delaware company with Clerky 2. Hire payroll company x 3. use this company for patents.

any info there?

From "Ask HN: What are your favorite entrepreneurship resources" https://news.ycombinator.com/item?id=15021659 :

> USA Small Business Administration: "10 steps to start your business." https://www.sba.gov/starting-business/how-start-business/10-...

> "Startup Incorporation Checklist: How to bootstrap a Delaware C-corp (or S-corp) with employee(s) in California" https://github.com/leonar15/startup-checklist

> FounderKit has reviews for Products, Services, and Software for founders: https://founderkit.com

... I've heard good things about Gusto for payroll, HR, and benefits through Guideline: https://gusto.com/product/pricing

[-]

A Self-Learning, Modern Computer Science Curriculum

Outstanding resource.

jwasham/coding-interview-university also links to a number of also helpful OER resources: https://github.com/jwasham/coding-interview-university

[-]

MVP Spec

> The criticism of the MVP approach has led to several new approaches, e.g. the Minimum Viable Experiment MVE[19] or the Minimum Awesome Product MAP[20].

https://en.wikipedia.org/wiki/Minimum_viable_product#Critici...

[-]

Can we merge Certificate Transparency with blockchain?

From "REMME – A blockchain-based protocol for issuing X.509 client certificates" https://news.ycombinator.com/item?id=18868540 :

""" In no particular order, there are a number of blockchain PKI (and DNS (!)) proposals and proofs of concept.

"CertLedger: A New PKI Model with Certificate Transparency Based on Blockchain" (2018) https://arxiv.org/pdf/1806.03914 https://scholar.google.com/scholar?q=related:LF9PMeqNOLsJ:sc...

"TABLE 1: Security comparison of Log Based Approaches to Certificate Management" (p.12) lists a number of criteria for blockchain-based PKI implementations:

- Resilient to split-world/MITM attack

- Provides revocation transparency

- Eliminates client certificate validation process

- Eliminates trusted key management

- Preserves client privacy

- Require external auditing

- Monitoring promptness

... These papers also clarify why a highly-replicated decentralized trustless datastore — such as a blockchain — is advantageous for PKI. WoT is not mentioned.

"Blockchain-based Certificate Transparency and Revocation Transparency" (2018) https://fc18.ifca.ai/bitcoin/papers/bitcoin18-final29.pdf

https://scholar.google.com/scholar?q=related:oEsKmJvdn-MJ:sc...

Who can update and revoke which records in a permissioned blockchain (or a plain old database, for that matter)?

Letsencrypt has a model for proving domain control with ACME; which AFAIU depends upon DNS, too. """

TLA references "Certificate Transparency Using Blockchain" (2018) https://eprint.iacr.org/2018/1232.pdf https://scholar.google.com/scholar?q="Certificate+Transparen...

[+]

> The main issue isn't the support and maintenance of a such distributed network,

Running a permissioned blockchain is nontrivial. "Just fork XYZ and call it a day" doesn't quite describe the amount of work involved. There's read latency at scale. There's merging things to maintain vendor strings,

> but its integration with current solutions

- Verify issuee identity

- Update (domain/CN/subjectAltName, date) index

- Update cached cert and CRL bundles

- Propagate changes to all clients

> and avoiding centralized middleware services that will weaken the schema described in the documents.

Eventually, a CDN will look desireable. IPFS may fit the bill, IDK?

[+]

google/trillian https://github.com/google/trillian

> Trillian is an implementation of the concepts described in the Verifiable Data Structures white paper, which in turn is an extension and generalisation of the ideas which underpin Certificate Transparency.

> Trillian implements a Merkle tree whose contents are served from a data storage layer, to allow scalability to extremely large trees.

[-]

Why Don't People Use Formal Methods?

Which universities teach formal methods?

- q=formal+verification https://www.class-central.com/search?q=formal+verification

- q=formal-methods https://www.class-central.com/search?q=formal+methods

Is formal verification a required course or curriculum competency for any Computer Science or Software Engineering / Computer Engineering degree programs?

Is there a certification for formal methods? Something like for engineer-status in other industries?

What are some examples of tools and [OER] resources for teaching and learning formal methods?

- JsCoq

- Jupyter kernel for Coq + nbgrader

- "Inconsistencies, rolling back edits, and keeping track of the document's global state" https://github.com/jupyter/jupyter/issues/333 (jsCoq + hott [+ IJavascript Jupyter kernel], STLC: Simply-Typed Lambda Calculus)

- TDD tests that run FV tools on the spec and the implementation

What are some examples of open source tools for formal verification (that can be integrated with CI to verify the spec AND the implementation)?

What are some examples of formally-proven open source projects?

- "Quark : A Web Browser with a Formally Verified Kernel" (2012) (Coq, Haskell) http://goto.ucsd.edu/quark/

What are some examples of projects using narrow and strong AI to generate perfectly verified software from bad specs that make the customers and stakeholders happy?

From reading though comments here, people don't use formal methods because: cost-prohibitive, inflexibile, perceived as incompatible with agile / iterative methods that are more likely to keep customers who don't know what formal methods are happy, lack of industry-appropriate regulation, and cognitive burden of often-incompatible shorthand notations.

[+]
[-]

Steps to a clean dataset with Pandas

To add to the three points in the article:

Data quality https://en.wikipedia.org/wiki/Data_quality

Imputation https://en.wikipedia.org/wiki/Imputation_(statistics)

Feature selection https://en.wikipedia.org/wiki/Feature_selection

datacleaner can drop NaNs, do imputation with "the mode (for categorical variables) or median (for continuous variables) on a column-by-column basis", and encode "non-numerical variables (e.g., categorical variables with strings) with numerical equivalents" with Pandas DataFrames and scikit-learn. https://github.com/rhiever/datacleaner

sklearn-pandas "[maps] DataFrame columns to transformations, which are later recombined into features", and provides "A couple of special transformers that work well with pandas inputs: CategoricalImputer and FunctionTransformer" https://github.com/scikit-learn-contrib/sklearn-pandas

Featuretools https://github.com/Featuretools/featuretools

> Featuretools is a python library for automated feature engineering. [using DFS: Deep Feature Synthesis]

auto-sklearn does feature selection (with e.g. PCA) in a "preprocessing" step; as well as "One-Hot encoding of categorical features, imputation of missing values and the normalization of features or samples" https://automl.github.io/auto-sklearn/master/manual.html#tur...

auto_ml uses "Deep Learning [with Keras and TensorFlow] to learn features for us, and Gradient Boosting [with XGBoost] to turn those features into accurate predictions" https://auto-ml.readthedocs.io/en/latest/deep_learning.html#...

[-]

Reahl – A Python-only web framework

kim0 | 2019-01-19 19:38:48 | 165 | # | ^
[+]
[+]

Before GWT, there was Wt framework (C++); and then JWt (Java), which do the server and clientsides (with widgets in a tree).

Wt: https://en.wikipedia.org/wiki/Wt_(web_toolkit)

JWt: https://en.m.wikipedia.org/wiki/JWt_(Java_web_toolkit)

GWT: https://en.wikipedia.org/wiki/Google_Web_Toolkit

Now we have Babel, ES YYYY, and faster browser release cycles.

[+]
[-]

Ask HN: How can you save money while living on poverty level?

I freelance remotely, making roughly $1200 a month as a programmer because I only work 10 hours maximum each week (limited by my contract). I share the apartment with my mom, and It's a section 8 so our rent contributions are based on the income we make. My contribution towards rent is $400 a month.

Although I make more money than my mom (she's of retirement age and only works 1-2 days a week), while I'm looking for more work I want to figure out how to move out and live more independently on only $1200 a month.

I need to live frugally and want to know what I can cut more easily. I own a used car (already paid in full), and pay my own car insurance, electricity, phone and internet. After all that I have about $400 left each month which can be eaten up by going out or some emergency funds.

More recently I had to pay for my new city parking sticker so that's $100 more in expenses this particular month. I would be satisfied just living in a far off town paying the same $400 a month, I feel my dollars would stretch further since I now get 100% more privacy for the same price.

On top of that this job is a contract job so I need to put money aside to pay my own taxes. This $1200 is basically living on poverty level. Any ideas to make saving work? Is it very possible for people in the US to still save while on poverty?

That's not a living wage (or a full time job). There are lots of job search sites.

Spending some time on a good resume / CV / portfolio would probably be a good investment with positive ROI.

Is there a nonprofit that you could volunteer with to increase your hireability during the other 158 hours of the week?

Or an online course with a credential that may or may not have positive ROI as a resume item?

Is there a code school in your city with a "you don't pay unless you land a full time job with a living wage and benefits" guarantee?

What is your strategy for business and career networking?

From https://westurner.github.io/hnlog/#comment-17894632 :

> Personal Finance (budgets, interest, growth, inflation, retirement)

Personal Finance https://en.wikipedia.org/wiki/Personal_finance

Khan Academy > College, careers, and more > Personal finance https://www.khanacademy.org/college-careers-more/personal-fi...

"CS 007: Personal Finance For Engineers" https://cs007.blog

https://reddit.com/r/personalfinance/wiki

[-]

Show HN: Generate dank mnemonic seed phrases in the terminal

From https://github.com/lukechilds/doge-seed :

> The first four words will be a randomly generated Doge-like sentence.

The seed phrases are fully valid checksummed BIP39 seeds. They can be used with any cryptocurrency and can be imported into any BIP39 compliant wallet.

> […] However there is a slight reduction in entropy due to the introduction of the doge-isms. A doge seed has about 19.415 fewer bits of entropy than a standard BIP39 seed of equivalent length.

[-]

Can you sign a quantum state?

> Abstract. Cryptography with quantum states exhibits a number of surprising and counterintuitive features. In a 2002 work, Barnum et al. argued informally that these strange features should imply that digital signatures for quantum states are impossible [6].

> In this work, we perform the first rigorous study of the problem of signing quantum states. We first show that the intuition of [6] was correct, by proving an impossibility result which rules out even very weak forms of signing quantum states. Essentially, we show that any non-trivial combination of correctness and security requirements results in negligible security.

> This rules out all quantum signature schemes except those which simply measure the state and then sign the outcome using a classical scheme. In other words, only classical signature schemes exist.

> We then show a positive result: it is possible to sign quantum states, provided that they are also encrypted with the public key of the intended recipient. Following classical nomenclature, we call this notion quantum signcryption. Classically, signcryption is only interesting if it provides superior efficiency to simultaneous encryption and signing. Our results imply that, quantumly, it is far more interesting: by the laws of quantum mechanics, it is the only signing method available.

> We develop security definitions for quantum signcryption, ranging from a simple one-time two-user setting, to a chosen-ciphertext-secure many-time multi-user setting. We also give secure constructions based on post-quantum public-key primitives. Along the way, we show that a natural hybrid method of combining classical and quantum schemes can be used to “upgrade” a secure classical scheme to the fully-quantum setting, in a wide range of cryptographic settings including signcryption, authenticated encryption, and chosen-ciphertext security.

"Quantum signcryption"

[-]

Lattice Attacks Against Weak ECDSA Signatures in Cryptocurrencies [pdf]

[+]

> Countermeasures. All of the attacks we discuss in this paper can be prevented by using deterministic ECDSA nonce generation [29], which is already implemented in the default Bitcoin and Ethereum libraries.

[-]

REMME – A blockchain-based protocol for issuing X.509 client certificates

[+]
[+]

In no particular order, there are a number of blockchain PKI (and DNS (!)) proposals and proofs of concept.

"CertLedger: A New PKI Model with Certificate Transparency Based on Blockchain" (2018) https://arxiv.org/pdf/1806.03914 https://scholar.google.com/scholar?q=related:LF9PMeqNOLsJ:sc...

"TABLE 1: Security comparison of Log Based Approaches to Certificate Management" (p.12) lists a number of criteria for blockchain-based PKI implementations:

- Resilient to split-world/MITM attack

- Provides revocation transparency

- Eliminates client certificate validation process

- Eliminates trusted key management

- Preserves client privacy

- Require external auditing

- Monitoring promptness

... These papers also clarify why a highly-replicated decentralized trustless datastore — such as a blockchain — is advantageous for PKI. WoT is not mentioned.

"Blockchain-based Certificate Transparency and Revocation Transparency" (2018) https://fc18.ifca.ai/bitcoin/papers/bitcoin18-final29.pdf

https://scholar.google.com/scholar?q=related:oEsKmJvdn-MJ:sc...

Who can update and revoke which records in a permissioned blockchain (or a plain old database, for that matter)?

Letsencrypt has a model for proving domain control with ACME; which AFAIU depends upon DNS, too.

[-]

California grid data is live – solar developers take note

> It looks like California is at least two generations of technology ahead of other states. Let’s hope the rest of us catch up, so that we have a grid that can make an asset out of every building, every battery, and every solar system.

+1. Are there any other states with similar grid data available for optimization; or any plans to require or voluntarily offer such a useful capability?

[-]

Why attend predatory colleges in the US?

> Why would people attend predatory colleges?

Why would people make an investment with insufficient ROI (Return on Investment)?

Insufficient information.

College Scorecard [1] is a database with a web interface for finding and comparing schools according to a number of objective criteria. CollegeScorecard launched in 2015. It lists "Average Annual Cost", "Graduation Rate", and "Salary After Attending" on the search results pages. When you review a detail page for an institution, there are many additional statistics; things like: "Typical Total Debt After Graduation" and "Typical Monthly Loan Payment".

The raw data behind CollegeScorecard can be downloaded from [2]. The "data_dictionary" tab of the "Data Dictionary" spreadsheet describes the data schema.

[1] https://collegescorecard.ed.gov

[2] https://collegescorecard.ed.gov/data/

Khan Academy > "College, careers, and more" [3] may be a helpful supplement for funding a full-time college admissions counselor in a secondary education institution

[3] https://www.khanacademy.org/college-careers-more

(I haven't the time to earn 10 academia.stackexchange points in order to earn the prestigious opportunity to contribute this answer to such a forum with threaded comments. In the academic journal system, journals sell academics' work (i.e. schema.org/ScholarlyArticle PDFs, mobile-compatible responsive HTML 5, RDFa, JSON-LD structured data) and keep all of the revenue).

"Because I need money for school! Next question. CPU: College Textbook costs and CPI: All over time t?!"

[-]

Ask HN: Data analysis workflow?

What kind of workflow do you employ when designing a data-flow or analyzing data?

Let me give a concrete example. For the past year, I have been selling stuff on the interwebs through two payment processors one of them being PayPal.

The selling process was put together with a bunch of SaaS hooking everything together through webhooks and notifications.

Now I need to step it that control and produce a proper flow to handle sign up, subscription and payment.

Before doing that I'm analyzing and trying to conciliate all transactions to make sure the books are OK and nothing went unseen. There lies the problem. I have data coming from different sources such as databases, excel files, CSV exports and some JSON files.

At first, I started dealing with it by having all the data in CSV files and trying to make sense of them using code and running queries within the code.

As I found holes in the data I had to dig up more data from different sources and it became a pain to continue with code. I now imported everything into Postgres and have been "debugging" with SQL.

As I advanced through the process I had to generate a lot of routines to collect and match data. I also have to keep all the data files around and organized which is very hard to do because I'm all over the place trying to find where the problem is.

How do you handle with it? What kind of workflow? Any best practices or recommendations from people who do this for a living?

Pachyderm may be basically what you're looking for. It does data version control with/for language-agnostic pipelines that don't need to always redo the ETL phase. https://www.pachyderm.io

Dask-ML works with {scikit-learn, xgboost, tensorflow, TPOT,}. ETL is your responsibility. Loading things into parquet format affords a lot of flexibility in terms of (non-SQL) datastores or just efficiently packed files on disk that need to be paged into/over in RAM. http://ml.dask.org/examples/scale-scikit-learn.html

Sklearn.pipeline.Pipeline API: {fit(), transform(), predict(), score(),} https://scikit-learn.org/stable/modules/generated/sklearn.pi...

https://docs.featuretools.com can also minimize ad-hoc boilerplate ETL / feature engineering :

> Featuretools is a framework to perform automated feature engineering. It excels at transforming temporal and relational datasets into feature matrices for machine learning.

The PLoS 10 Simple Rules papers distill a number of best practices:

"Ten Simple Rules for Reproducible Computational Research" http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fj...

“Ten Simple Rules for Creating a Good Data Management Plan” http://journals.plos.org/ploscompbiol/article?id=10.1371/jou...

In terms of the scientific method, a null hypothesis like "there is no significant relation between the [independent and dependent] variables" may be dangerously unprofessional p-hacking and data dredging; and may result in an overfit model that seems to predict or classify the training and test data (when split with e.g. sklearn.model_selection.train_test_split and a given random seed).

One of these days (in the happy new year!) I'll get around to updating these notes with the aforementioned tools and docs: https://wrdrd.github.io/docs/consulting/data-science#scienti...

IDK what https://kaggle.com/learn has specifically in terms of analysis workflow? Their docker containers have very many tools configured in a reproducible way: https://github.com/Kaggle/docker-python/blob/master/Dockerfi...

[-]

Ask HN: What is your favorite open-source job scheduler

Too many business scripts rely on cron(8) to run. Classic cron cannot handle task duration, fail (only with email), same-task piling, linting, ...

So what is your favorite open-source, easy to bundle/deploy job scheduler, that is easy to use, has logging capacity, config file linting, and can handle common use-cases : kill if longer than, limit resources, prevent launching when previous one is nor finished, ...

systemd-crontab-generator may be usable for something like linting classic crontabs? https://github.com/systemd-cron/systemd-cron

Systemd/Timers as a cron replacement: https://wiki.archlinux.org/index.php/Systemd/Timers#As_a_cro...

Celery supports periodic tasks:

> Like with cron, the tasks may overlap if the first task doesn’t complete before the next. If that’s a concern you should use a locking strategy to ensure only one instance can run at a time (see for example Ensuring a task is only executed one at a time).

http://docs.celeryproject.org/en/latest/userguide/periodic-t...

[-]

How to Version-Control Jupyter Notebooks

tosh | 2018-12-22 06:53:46 | 164 | # | ^

Mentioned in the article: manual nbconvert, nbdime, ReviewNB (currently GitHub only), jupytext.

Jupytext includes a bit of YAML in the e.g. Python/R/Julia/Markdown header. https://github.com/mwouts/jupytext

[+]
[+]

Teaching and Learning with Jupyter (A book by Jupyter for Education)

[+]
[+]
[+]
[-]

Margin Notes: Automatic code documentation with recorded examples from runtime

[+]
[+]
[+]

1. sys.settrace() for {call, return, exception, c_call, c_return, and c_exception}

2. Serialize as/to doctests. Is there a good way to serialize Python objects as Python code?

3. Add doctests to callables' docstrings with AST

Mutation testing tools may have already implemented serialization to doctests but IDK about docstring modification.

... MOSES is an evolutionary algorithm that mutates and simplifies a combo tree until it has built a function with less error for the given input/output pairs.

[-]

Time to break academic publishing's stranglehold on research

[+]
[+]
[+]
[+]
[+]

https://hypothes.is supports threaded comments on anything with a URI; including PDFs and specific sentences or figures thereof. All you have to do is register an account and install the browser extension or include the JS in the HTML.

It's based on open standards and an open platform.

W3C Web Annotations: http://w3.org/annotation

About Hypothesis: https://web.hypothes.is/about/

[-]

Ask HN: How can I learn to read mathematical notation?

There are a lot of fields I'm interested in, such as machine learning, but I struggle to understand how they work as most resources I come across are full of complex mathematical notation that I never learned how to read in school or University.

How do you learn to read this stuff? I'm frequently stumped by an academic paper or book that I just can't understand due to mathematical notation that I simply cannot read.

[+]

There are a number of Wikipedia pages which catalog various uses of symbols for various disciplines:

Outline_of_mathematics#Mathematical_notation https://en.wikipedia.org/wiki/Outline_of_mathematics#Mathema...

List_of_mathematical_symbols https://en.wikipedia.org/wiki/List_of_mathematical_symbols

List_of_mathematical_symbols_by_subject https://en.wikipedia.org/wiki/List_of_mathematical_symbols_b...

Greek_letters_used_in_mathematics,_science,_and_engineering https://en.wikipedia.org/wiki/Greek_letters_used_in_mathemat...

Latin_letters_used_in_mathematics https://en.wikipedia.org/wiki/Latin_letters_used_in_mathemat...

For learning the names of symbols (and maybe also their meaning as conventially utilized in a particular field at a particular time in history), spaced repetition with flashcards with a tool like Anki may be helpful.

For typesetting, e.g. Jupyter Notebook uses MathJax to render LaTeX with JS.

latex2sympy may also be helpful for learning notation.

… data-science#mathematical-notation https://wrdrd.github.io/docs/consulting/data-science#mathema...

[-]

New law lets you defer capital gains taxes by investing in opportunity zones

[+]
[+]