r - Grouping very small numbers (e.g. 1e-28) and 0.0 in data.table v1.8.10 vs v1.9.2 -


i noticed frequency tables created data.table in r seem not distinguish between small numbers , zero? can change behavior or bug?

reproducible example: 

>library(data.table)    dt <- data.table(c(0.0000000000000000000000000001,2,9999,0))     test1 <- as.data.frame(unique(dt[,v1]))    test2 <-  dt[, .n, = v1]

as can see, frequency table (test2) not recognize differences between 0.0000000000000000000000000001 , 0 , put both observations in same class.

data.table version: 1.8.10
r: 3.02

it worth reading r faq 7.31 , thinking accuracy of floating point represenations.

i can't reproduce in current cran version (1.9.2). using 

r version 3.0.3 (2014-03-06) platform: x86_64-w64-mingw32/x64 (64-bit)

my guess change in behaivour related news item.

o numeric data still joined , grouped within tolerance before instead of tolerance being sqrt(.machine$double.eps) == 1.490116e-08 (the same base::all.equal's default) significand rounded last 2 bytes, apx 11 s.f. more appropriate large (1.23e20) , small (1.23e-20) numerics , faster via simple bit twiddle. few functions provided 'tolerance' argument wasn't being passed through has been removed. aim add global option (e.g. 2, 1 or 0 byte rounding) in future release.

update matt

yes deliberate change in v1.9.2 , data.table distinguishes 0.0000000000000000000000000001 0 (as user3340145 rightly thought should) due improved rounding method highlighted above news.

i've added for loop test rick's answer test suite.

btw, #5369 implemented in v1.9.3 (although neither of these needed question) :

o bit64::integer64 works in grouping , joins, #5369. james sams highlighting upcs.

o new function setnumericrounding() may used reduce 1 byte or 0 byte rounding when joining or grouping columns of type 'numeric', #5369. see example in ?setnumericrounding , news item v1.9.2. getnumericrounding() returns current setting.

notice rounding (as v1.9.2) accuracy of significand; i.e. number of significant figures. 0.0000000000000000000000000001 == 1.0e-28 accurate 1 s.f., new rounding method doesn't group 0.0.

in short, answer question : upgrade v1.8.10 v1.9.2 or greater.


Comments

Post a Comment

Popular posts from this blog

Android layout hidden on keyboard show -

i have strange problem layout. when tap on edittext , keyboard shown edittext stay on bottom , when tap on to hide keyboard , keyboard hides , edittext go should when want input text. when tap again, keyboard shown , edittext goes down behind keyboard... <relativelayout android:layout_width="match_parent" android:layout_height="match_parent" > <scrollview android:layout_width="match_parent" android:layout_height="fill_parent" android:above="@+id/rl_commands"> <textview android:layout_width="match_parent" android:layout_height="wrap_content"/> </scrollview> <relativelayout android:layout_width="match_parent android:layout_height="24dp" android:id="@+id/rl_commands" android:alignparentbottom="true">
Read more

google app engine - 403 Forbidden POST - Flask WTForms -

i have flask application running on google app engine. trying submit form built using wtform , keep getting following error. 403 forbidden: don't have permission access requested resource. either read-protected or not readable server. location.html (part of code) <form method=post action="/home/location"> .... <button type="submit" class="btn btn-primary" value="submit"> directions </button> </form> main.py @app.route('/home/location', methods=['post', 'get']) def location(): form = cfcdirections.direction(request.form) print(request.method) if request.method == 'post' , form.validate(): print("after if") directions = cfcdirections() street_no = directions.no street = directions.street suburb = directions.suburb postcode = directions.postcode
Read more

c - Why would PK11_GenerateRandom() return an error -8023? -

i looking through internet trying find source of pk11_generaterandom() function see why function fail. have program uses function when moved new flavor of linux, fails after forking ( fork() ) since not believe there problem nss, suspect doing incorrectly disregarded in older versions of linux new 1 there issue. the openssl package same on 'good' , 'bad' server: openssl 0.9.8e-fips-rhel5 01 jul nss rpm differs though. 'good' has nss-3.12.2.0-2.el5 and bas has version nss-3.15.3-4.el5_10 the 'good' server uses quite obsolete linux: linux 2.6.18-128.el5 #1 smp wed jan 21 08:45:05 est 2009 x86_64 x86_64 x86_64 gnu/linux enterprise linux enterprise linux server release 5.3 (carthage) red hat enterprise linux server release 5.3 (tikanga) the 'bad' server newer: linux bad 2.6.18-371.4.1.el5 #1 smp wed jan 29 11:05:49 pst 2014 x86_64 x86_64 x86_64 gnu/linux oracle linux server release 5.10 red hat enterprise linux server release
Read more