-
Notifications
You must be signed in to change notification settings - Fork 32
/
ChangeLog
executable file
·3725 lines (2558 loc) · 145 KB
/
ChangeLog
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
** denotes quite substantial/important changes
*** denotes really big changes
Liable to change in future release:
- method="GCV.Cp" as default in gam. Default will
change to "REML". (1.8-24, June 2018)
Currently deprecated and liable to be removed:
- gam.fit and full.score (1.9-0)
Issues:
* openblas 0.3.x x<7 is not thread safe if itself compiled for single thread
use and then called from multiple threads (unlike the reference BLAS, say).
0.2.20 appears to be OK. For 0.3.x x>6 make USE_THREAD=0 USE_LOCKING=1
to make openblas ensures thread safety.
* t2 in bam(...,discrete=TRUE) - not treated as tensor products at
present, and reparameterization needs checking (also for bam).
* bam(...,discrete=TRUE) has no option to use the identical
discretizaton to that used in fitting when predicting.
1.9-1
* Revised bam discrete code to allow indexing vectors to be long vectors,
thereby allowing much larger datasets to be used. n limit is still
.Machine$integer.max.
* Revised C code underlying mvn family to allow larger problems, via use of
R long vectors.
* Revised gfam to work correctly with bam(...,discrete=FALSE). Subsetting
in bgam.fit was incorrect for gfam. Thanks to Dave Miller for reporting it.
* Correction of ziplss residual computation - incorrect probability of non
zero used. Thanks to Xiao Liang.
* correction of ginla integration weights (correcting typo from Rue et al.
2009). Thanks to Paul Van Dam-Bates.
* C code modification to remove some new compiler warnings.
1.9-0
*** NCV smoothing parameter estimation now available for most models. See ?NCV.
*** 'gfam' allows reponse variables to be from several different families.
* Restriction that number of coefficients must be fewer than number of data
removed.
* Fix to tensor product constructor to allow matrix factor arguments to be
handled correctly (previously failed). Thanks to Dave Miller.
* removed argument 'pers' from 'plot.gam' (deprecated since 1.8-23, Nov 2017).
* removed "nlm.fd" optimizer (deprecated since 1.8-19, Sept 2017).
* removed discrete method prediction from discrete bam objects fitted
before 1.8-32 (deprecated since then).
* 'multinom' variance fix for K>=3 cases. A counter was initialized in the wrong
place. Thanks to Max Goplerud for finding and reporting this.
* NCV variance estimation improvements + na handling.
* 'in.out' argument added for 'bam'.
* slight internal change to mgcv:::get.var to enable user defined functions
in arguments of smooths to work.
* smooth2random tweak to guarantee same eigenvector sign everytime for same
problem (needed by some developers who don't want to carry the original
transform around).
* Some unregistered methods fixes.
* tests directory removed, as check time was tripping up every CRAN submission.
* coxph memory error fix - could make bfgs fitting fail.
1.8-42
* One remaining old style C declaration fixed.
1.8-41
** 'cnorm' family added for left, right, interval or un censored Gaussian
data. Useful for log normal Accelerated Failure Time models, Tobit
regression, rounded data etc.
** 'sz' factor smooth interaction class added for implementing models with
main effect smooths and difference smooths for levels of a factor.
See ?factor.smooth.
** 'NCV' smoothing parameter method added, but still experimental.
* replacment of some old (K&R) style C function declarations.
* mono.con corrected for cases with upper bounds. Thanks to Sean Wu.
* plot.gam(...,seWithMean=TRUE) modified to only include mean uncertainty
for the linear predictor of which the smooth is a part, when there are
multiple linear predictors. Thanks to Gavin Simpson.
* modifications of sparce matrix coercions, to avoid deprecated direct
coercions. as(as(as(a, "dMatrix"), "generalMatrix"), "XsparseMatrix") in
place of as(a,"dgXMatrix") where 'X' is 'C' or 'T'. Actually this requires
Matrix 1.4-2 to work (will be added to dependencies in future).
* vis.gam now deals properly with models with more than one linear predictor.
* slight change in bgam.fitd to check scale parameter estimate converged when
using bam(...,discrete=TRUE), otherwise scale could be wrong for all fixed
smoothing parameters.
* predict.gam modified so that 'terms' and 'exclude' control all terms, smooth
or parametric, in the same way. Including the "(Intercept)" term.
* Warnings from 'model.matrix' suppressed in 'terms2tensor' called by e.g.
predict.bamd. There is a warning if any extra contrasts are supplied to
model matrix that do not relate to a term in the model (which contradicts
the documentation). Doc vs code bug report also filed.
* Fix of broken rank deficiency handling in gam.fit5. Thanks to Cesko Voeten.
* trind.generator modified to allow return of index functions in place
of index arrays.
* summary.gam (recov/reTest) modified to deal with 'fs' smooths fitted using
'gamm'.
* gam.fit4 convergence testing improved and bug fix in computation of dVkk
matrix used to check for converged 'infinite' smoothing parameters in bfgs.
1.8-40
* Small gam.fit5 convergence change to reduce chance of repeating step
failures.
* ExtractData fix avoiding assumption that 'xt' argument to a smooth is always
a list (could cause setup failure with e.g. fs smooth). Thanks Keith Woolner.
* Minor changes to mat.c and tprs.c to allow DEFS = -DSTRICT_R_HEADERS build.
1.8-39
* gam.fit5 convergence logic tightened up, to avoid pointlessly lengthy
iterations when can't meet convergence tolerance. Faster, more reliable,
but also generates more warnings when tolerances not met.
* gam.fit3 convergence test modified to more reliable version.
* Some modification of warning handling to only print inner optimization
warnings if they occur at final call to inner optimizer.
* newton and bfgs optimizers reset inner loop tolerance to 1e-2 of outer
loop tolerance if it is larger than this, to avoid inner loop being too
inaccurate for outer tolerance.
* minor R 4.2.0 C compatibility change.
1.8-38
* uniquecombs fix to reduce memory footprint for text data.
1.8-37
* minor update for clang compatibility.
* PredictMat fix of bug in which dimensions could be dropped for 1 row matrix
as part of constraint handling (leading to disaster in predict.bamd).
Thanks to Shawn Ligocki.
1.8-36
* 'fs' smooth construction modified to (approximately) orthogonalise
smoothing penalty penalized and unpenalized bases. This makes the
assumption that the associated variance components are independent
more natural. Thanks to Matteo Fasiolo and Harald Bayaan for showing
how the original construction could be problematic.
* correction to 'fs' smooth handling in gamm to allow correlation structures
to be used in the same model without cuasing an error.
* AIC calculation modified for non-Gaussian families with a scale parameter,
so that the model estimated scale parameter is used, rather than the
deviance estimator employed with glm exponential families. This matters for
avoiding substantial bias with very low mean count data (particulalry 'tw'
and 'Tweedie' families).
* ziP family response scale se fix for case when b>0 (thanks to Mark Donoghoe)
* prediction on response scale could fail for models fitted by
bam(...,discrete=TRUE) - fixed.
* bug fix to multi-model anova.gam code to deal properly with extended family
scale parameters (still not the recommended way of testing!)
1.8-35
* Fix to bug in Fletcher scale parameter estimate with weighted data (e.g.
quasibinomial with n>1). Gaussian case not affected. Thanks to John
Maindonald.
* Italian translation updated thanks to Daniele Medri
* German translation updated thanks to Detlef Steuer
* French translation updated thanks to Philippe Grosjean
* gumbls initialization improved.
* Modification so that smooths can insist on not being reparameterized when
called using default bam methods, by having a 'repara' element set to FALSE
in the object returned by the smooth constructor.
* Modification so that smooths can have the summation convention for matrix
arguments turned off, by having an 'xt$sumConv' element set to FALSE in the
'xt' argument of 's'.
1.8-34
* 'gam.mh' proposal df typo fix and help file '%*%' -> '\%*\*' fix. Thanks
Len Thomas.
* various random number generation calls in rd functions in families could
fail if simulating one point - fixed.
* gp smooths can now be specified to be strictly stationary (see ?gp.smooth)
* fix to predict.bamd bug in handling fixed effect contrasts (wrong contrast
could be used for prediction, if non-default used in fitting). Thanks to
Jalal Al-Tamimi.
* predict.bamd modified to catch case where all prediction data is NA
* predict.gam modified to allow NA to be a factor level.
1.8-33
* 'inline' -> 'static inline' error fix - caused installation failure on some
platforms.
1.8-32
** bam(...,discrete=TRUE) now uses discretization on the parametric model
components as well as the smooths.
* 'gumbls' family added for Gumbel location scale models.
* 'shash' location scale and shape family added.
* psum.chisq function added to compute c.d.f. of weighted sums of chi-squared
r.v.s using method of Davies, 1980. This is now used for computing p-values
in summary.gam in place of the Liu et al, 2009 approximation.
* score and Schoenfeld residuals added for cox.ph to facilitate PH assumption
checking (see ?cox.ph).
* added 'gam.mh' for posterior dampling from models fitted with 'gam'.
* 'ocat' upgraded to not ignore weights.
* added multivariate t and Gaussian densities and multivariate t generation.
* fix to predict.bamd to correct handling of `terms' and `exclude' arguments.
* A recent lme change broke gamm offset handling. Fixed.
Thanks Aaron Benjamin Shev.
* dgesvd LAPACK calls replaced by faster dgesdd, as some versions of MKL
BLAS/LAPACK have broken dgesvd.
* Fix to derivative w.r.t. scale parameter calculation for extended families
('tw' is the only one at present) when using 'ML' smoothing parameter
estimation. Could lead to step failure with 'tw'. Thanks Kevin Hawkshaw.
* Fix to prediction/fake formula environment that could cause failure of
gam/bam called from a function. Thanks to Duncan Murdoch.
* Fix to smoothing parameter uncertainy correction for general families.
* Much cleaner design for the discretization of covariates and associated
discretization index matching in bam (predict.bamd and discrete.mf).
* Fix to jagam logic for identifying separable penalties which led to
wrong partioning of penalties for 'fs' smooths used with jagam. Thanks to
Chris Jackson for spotting the problem.
* Internal Sl list handling changes.
* tensor.prod.model.matrix now also works with sparse matrices of class
"dgCMatrix".
1.8-31
* fix in initalization in gammPQL
* fix of some C routines of type void in place of SEXP called by .Call.
1.8-30
* anova.gam now uses GLRT for multi-model comparisons in both extended and
general family cases.
* Fix to bug in bam(...,discrete=TRUE) offset handling introduced in 1.8-29,
which corrupted offset (and generated numerous warnings). Also fixes a
less obvious bug introduced at the same time in predict.gam which could get
the offset wrong when predicting with such models. Thanks to Brian
Montgomery and Sean Wilson.
* Fix to problem in pen.reg (used to initialize location scale models in
particular), which could lead to initialization failure for lightly
penalized models. Thanks Matteo Fasiolo.
* Fix to predict.bamd handling of 'terms' and 'exclude' for models fit by
bam(...,discrete=TRUE).
* Work around in predict.gam for a spurious model.matrix warning when a
contrasts.arg relates to a variable in data not required by object.
* Fix to gammals family which had 'd2link' etc defined with argument 'eta'
instead of the intended 'mu'. Thanks to Jim Stagge.
* 'in.out' modified to allow boundaries specified exactly as for a soap film
smoother.
* soap film smoother constructor modified to check knots are within boundary
on entry and drop those that are not, with a warning. Also halts if it
detects that basis setup is catastrophically ill-conditioned.
1.8-29
* gammPQL modified to use standard GLM IRLS initilization (rather than
glmmPQL method) to improve convergence of `gamm' fits.
* bam(...,discrete=TRUE) now drops rownames from parametric model matrix
components, to save substantial memory.
* All BLAS/LAPACK calls from C now explicitly pass hidden string length
arguments to avoid breakage by recent gfortran optimizations (stack
corruption causing BLAS/LAPACK to return error code).
* predict.gam bug fix - parameteric interaction terms could be dropped for
type="terms" if there were no smooths. knock on was that they were also
dropped for all bam(...,discrete=TRUE) fits. (Thanks Justin Davis.)
* bam(...,discrete=TRUE) indexing bug in setup meant that models containing
smooths with matrix arguments and other smooths with factor by variable
would fail at the setup stage.
* gam.fit4 initial divergence bug fix.
* Gamma location-scale family 'gammals' added. See ?gammals.
* row-wise Kronecker product operator %.% added for convenience.
* changes to general families to allow return of first deriv of penalized
Hessian component more easily.
* ocat bug fix. Response scale prediction was wrong - it ignored the estimated
thresholds. Thanks to Fabian Scheipl.
* bam deviance could be wrongly returned, leading to 100% explained
deviance. Fixed.
1.8-28
* fix of obscure sp naming bug.
* changed some contour default colours from green to blue (they overlay
heatmaps, so green was not clever).
* Tweedie likelihood evaluation code made slightly more robust - for a model
with machine zero scale parameter estimate it could segfault, as series
maximum location could then overflow integer storage. Fixed + upper limit
imposed on series length (warning if it's not enough).
1.8-27
** Added routine 'ginla' for fully Bayesian inference based on an integrated
nested Laplace approximation (INLA) approach. See ?ginla.
* Tweedie location scale family added: 'twlss'.
* gam.fit5 modified to distinguish more carefully between +ve semi definite
and +ve definite. Previously could fail, claiming indefiniteness when it
should not have. Affects general families.
* bam was ignoring supplied scale parameter in extended family cases - fixed.
* work around in list formula handling for reformulate sometimes setting
response to a name in place of a call.
* preinitialize in general families is now a function, not an expression. See
cox.ph for an example.
* added routine cholup for rank one modification of Cholesky factor.
* two stage gam/bam fitting now allows 'sp' to be modified.
* predict.gam could fail with type="response" for families requiring the
response to be provided in this case (e.g. cox.ph). Fixed.
* sp.vcov defaults to extracting edge corrected log sp cov matrix, if
gam(...,gam.control(edge.control=TRUE)) used for fitting.
* gam(...,gam.control(edge.correct=TRUE)) could go into infinite loop if
sp was effectively zero. Corrected.
1.8-26
* LINPACK dependency removed.
* Added service routine choldrop to down date a Cholesky factor on row/col
deletion.
* liu2 had a check messed up when vectorized. Fix to stop vector being
checked for equality to zero.
1.8-25
** bam(...,discrete=TRUE) methods improved. Cross products now usually faster
(can be much faster) and code can now make better use of optimised BLAS.
* fix to 'fs' smooth.construct method and smooth2random method, to allow
constructor to be called without a "gamm" atribute set on the smooth spec
but still get a sensible result from smooth2random (albeit never using
sarse matrices). Useful for other packages using constructors and
smooth2random, for 'fs' smooths.
* The mrf smooth constructor contained an obsolete hack in which the term
dimension was set to 2 to avoid plotting when used as a te marginal. This
messed up side constraints for terms where a mrf smooth was a main effect
and te marginal. Fixed.
* extract.lme.cov/2 documentation modified to cover NA handling properly, and
routines modified to not require data to be supplied.
* fix of efsudr bug whereby extended families with no extra parameters to
estimate could give incorrect results when using optimer="efs" in 'gam'.
* negbin() corrected - it was declaring the log link to be canonical, leading
to poor convergence and slight misfit.
* predict.bam(...,discrete=TRUE) now handles na.action properly, rather than
always dropping NAs.
* Fix of very obscure bug in which very poor model of small dataset could
end up with fewer `good' data than coefs, breaking an assumption of C code
and segfaulting.
* fix of null deviance computation bug introduced with extended families in
bam - null deviance was wrong for non-default methods.
* liu2 modified to deal with random effects estimated to be exactly 0, so
that summary.gam does not fail in this case.
1.8-24
* Extended Fellner Schall optimizer now avaialable for all families with 'gam'
using gam(...,optimizer="efs").
* Change to default behaviour of plot.gam when 'seWithMean=TRUE', and of
predict.gam when 'type="iterms"'. The extra uncertainty added to CIs or
standard errors now reflects the uncertainty in the mean in all other model
terms, not just the uncertanity in the mean of the fixed effects as before.
See ?plot.gam and ?predict.gam (including for how to get the old behaviour).
* 're' smooths can now accept matrix arguments: see ?linear.functional.terms.
* cox.ph now allows an offset to be provided.
* Fix in smoothCon for bug in case in which only a single coefficient is
involved in a sum-to-zero constraint. Could cause failure in e.g. t2 with
cc marginal.
* Model terms s, te etc are now always evaluated in mgcv workspace explicitly
to avoid masking problems in obscure circumstances.
* 'mrf' smooth documentation modified to make it clearer how to specify 'nb',
and code modified so that it is now possible to specify the neighbour
structure using names rather than indices.
* 'bfgs' fix to handle extended families.
* plot.gam modified to only prompt (via devAskNewPage) for a new page after
the first page is used up.
* export 'k.check'.
* Fix to 'Rrank'. Previously a matrix R with more columns than rows could
cause a segfault.
* Fix to non-finite likelihood handling in gam.fit5.
* Fix in bgam.fitd to step reduce under indefinite deviance and to ensure
penalty evaluation is round off negative proof.
* newton slighty modified to avoid (small) chance of all sp's being dropped
for indef likelihood.
1.8-23
* default plot methods added for smooths of 3 and 4 variables.
* The `gamma' control parameter for gam and bam can now be used with RE/ML
smoothness selection, not just GCV/AIC. Essentially smoothing parameters
are chosen as if the sample size was n/gamma instead of n.
* The "bs" basis now allows multiple penalties of different orders on the same
spline. e.g. s(x,bs="bs",m=c(3,2,0)). See ?b.spline.
* bam(...,discrete=TRUE) can now use the smooth 'id' mechanism to link
smoothing parameters, but note the method constraint that the linked bases
are not forced to be identical in this case (unlike other fitting methods).
* summary.gam now allows random effects tests to be skipped (in some large
models the test is costly and uninteresting).
* 'interpret.gam0' modified so that masked 's', 'te', etc from other packages
does not cause failure.
* coxph fix of prediction bug introduced with stratified model (thanks
Giampiero Marra)
* bam(...,discrete=TRUE) fix to handle nested matrix arguments to smooths.
* bam(...,discrete=TRUE) fix to by variable handling with fs and re smooths
which could fail during re-representation as tensor smooths (for
discretization purposes).
* bam extended family extension had introduced a bug in null deviance
computation for Gaussian additive case when using methods other than fREML
or GCV.Cp. Fixed.
* bam(...,discrete=TRUE) now defaults to discrete=FALSE if there are no
smooths, rather than failing.
* bam was reporting wrong family under some smoothing parameter selection
methods (not default).
* null deviance computation improved for extended families. Previous version
used an approximation valid for most families, and corrected the rest - now
replaced with exact computations for all cases.
* scat initialization tweaked to avoid -ve def problems at start.
* paraPen handling in bam was broken - fixed.
* slight adjustment to sp initialization for extended families - use observed
information in weights if possible.
1.8-22
* Fix of bug whereby testing for OpenMP and nthreads>1 in bam, would fail if
OpenMP was missing.
1.8-21
* When functions were added to families within mgcv some very large
environments could end up attached to those functions, for no good reason.
The problem originated from the dispatch of the generic 'fix.family.link'
and then propagated via fix.family.var and fix.family.ls. This is now avoided,
resulting in smaller gam objects on disk and lower R memory usage.
Thanks to Niels Richard Hansen for uncovering this.
* Another bug fix for prediction from discrete fit bam models with an offset,
this time when there were more than 50000 data. Also fix to bam fitting when
the number of data was an integer multiple of the chunk size + 1.
* check.term was missing a 'stop' so that some unhandled nesting structures
in bam(...,discrete=TRUE) failed with an unhelpful error, instead of a
helpful one. Fixed.
1.8-20
* bam(,discrete=TRUE) could produce garbage with ti(x,z,k=c(6,5),mc=c(T,F))
because tensor re-ordering for efficiency failed to re-order mc (this is
a *very* specialist bug!). Thanks to Fabian Scheipl.
* plot(...,residuals=TRUE) weighted the working residuals by the sqrt working
weights divided by the mean sqrt working weight. The standardization by the
mean sqrt weight was non standard and has been removed.
* Fix to bad bug in bam(...,discrete=TRUE) offset handling, and predict.bamd
modified to avoid failure predicting with offset. Thanks to Paul Shearer.
* fix of typo in bgam.fit, which caused failure of extended families
when dataset larger than chunk size. Thanks Martijn Wieling.
* bam(...,discrete=TRUE)/bgam.fitd modified to use fisher weights with
extended families if rho!=0.
1.8-19
** bam() now accepts extended families (i.e. nb, tw, ocat etc)
* cox.ph now allows stratification (i.e. baseline hazard can differ between
groups).
* Example code for matched case control in ?cox.ph was just plain wrong. Now
fixed. Thanks to Patrick Farrell.
* bam(...,discrete=TRUE) tolerance for judging whether smoothing parameters
are on boundary was far too low, so that sps could become so large that
numerical instability set in. Fixed. Thanks to Paul Rosenfield.
* p.type!=0 removed in summary.gam (previously deprecated)
* single penalty tensor product smooths removed (previously deprecated).
* gam(...,optimizer="perf") deprecated.
* extra divergence check added to bam gam default gam fitting (similar to
discrete method).
* preinitialize and postproc components of extended families are now functions,
not expressions.
* coefficient divergence check was missing in bam(...,discrete=TRUE) release
code - now fixed.
* gaulss family link derivatives modified to avoid overflow. Thanks to
Kristen Beck for reporting the problem.
* redundant 'n' argument removed from extended family 'ls' functions.
* Convergence checking can step fail earlier in fast.REML.fit. If trial step
is no improvement and equal to previous best (to within a tolerance), then
terminate with step failure after a few step halvings if situation persists.
Thanks to Zheyuan Li for reporting problem.
1.8-18
* Tweak to 'newton' to further reduce chance of false convergence at
indefinite point.
* Fix to bam.update to deal with NAs in response.
* 'scat' family now takes a 'min.df' argument which defaults to 3. Could
otherwise occasionally have indefinite LAML problems as df headed towards 2.
* Fix to `gam.fit4' where in rare circumstances the PIRLS iteration could
finish at an indefinite point, spoiling implicit differentiation.
* `gam.check' modified to fix a couple of issues with `gamm' fitted models, and
to warn that interpretability is reduced for such models.
* `qq.gam' default method slight modification to default generation of reference
quantiles. In theory previous method could cause a problem if enough
residuals were exactly equal.
* Fix to `plot.mrf.smooth' to deal with models with by variables.
* `plot.gam' fix to handling of plot limits when using 'trans' (from 1.8-16
'trans' could be applied twice).
* `plot.gam' argument 'rug' now defaults to 'NULL' corresponding to 'rug=TRUE'
if the number of data is <= 10000 and 'rug=FALSE' otherwise.
* bam(...,discrete=TRUE) could fail if NAs in the smooth terms caused data rows
to be dropped which led to parametric term factors having unused levels
(which were then not being dropped). Fixed (in discrete.mf).
* bam(...,discrete=TRUE,nthreads=n) now warns if n>1 and openMP is not
available on the platform being used.
* Sl.addS modified to use C code for some otherwise very slow matrix
subset and addition ops which could become rate limiting for
bam(...,discrete=TRUE).
* Parallel solves in Sl.iftChol can speed up bam(...,discrete=TRUE) with
large numbers of smoothing/variance parameters.
* 'gamm' now warns if called with extended families.
* disasterous 'te' in place of 'ti' typo in ?smooth.terms fixed thanks to
John McKinlay.
* Some `internal' functions exported to facilitate quantile gam methods
in separate package.
* Minor fix in gam.fit5 - 1 by 1 matrix coerced to scalar, to prevent failure
in some circumstances.
1.8-17
* Export gamlss.etamu, gamlss.gH and trind.generator to facilitate user
addition of new location-scale families.
* Re-ordering of initialization in gam.fit4 to avoid possible failure of
dev.resids call before initialization.
* trap in fast.REML.fit for situation in which all smoothing parameters
satisfy conditions for indefinite convergence on entry, with an
immediate warning that this probably indicates iteration divergence (of bam).
* "bs" basis modified to allow easier control of the interval over which the
spline penalty applies which in turn allows more sensible control of
extrapolation behaviour, when this is unavoidable.
* Fix in uniquecombs - revised faster code (from 1.8-13) could occasionally
generate false matches between different input combinations for integer
variables or factors. Thanks to Rohan Sadler for reporting the issue that
uncovered this.
* A very bad initial model for uninformative data could lead to a negative
fletcher estimate of the scale parameter and optimizer failure - fixed.
* "fREML" allowed in sp.vcov so that it works with bam fitted models.
* 2 occurances of 'return' replaced by (correct) return().
1.8-16
* slightly improved intial value heuristics for overlapping penalties in
general family case.
* 'ocat' checks that response is numeric.
* plot.gam(...,scale=-1) now changes scale according to 'trans' and 'shift'.
* newton optimizer made slightly more cautious: contracts step if reduction
in true objective too far different from reduction predicted by quadratic
approximation underlying Newton step. Also leaves parameters unchanged
in Newton step while their grad is less than 1% of max grad.
* Fix to Fisher weight computation in gam.fit4. Previously a weight could
(rarely) evaluate as negative machine prec instead of zero, get passed to
gdi2 in C code, generate a NaN when square rooted, resulting in a NaN passed
to the LAPACK dgeqp3 routine, which then hung in a non-interuptable way.
* Fix of 'sp' argument handling with multiple formulae. Allocation to terms
could be incorrect.
* Option 'edge.correct' added to 'gam.control' to allow better correction
of edge of smoothing parameter space effects with 'gam' when RE/ML used.
* Fix to setting of penalty rank in smooth.construct.mrf.smooth.spec.
Previously this was wrong, which could cause failure with gamm if the
penalty was rank deficient. Thanks Paul Buerkner.
* Fix to Vb.corr call from gam.fit3.post.proc to ensure that sp not
dropped (wrongly treated as scale estimate) when P-REML or P-ML used.
Could cause failure depending on BLAS. Thanks Matteo Fasiolo.
* Fix in gam.outer that caused failure with "efs" optimizer and fixed sps.
* Fix to `get.var' to drop matrix attributes of 1 column matrix variables.
* Extra argument added to `uniquecombs' to allow result to have same row
ordering regardless of input data ordering. Now used by smooth constructors
that subsample unique covariate values during basis setup to ensure
invariance to data re-ordering.
* Correction of scaling error in spherical correlation structure GP smooth.
* qf and rd functions for binomial family fixed for zero n case.
1.8-15
* Fix of survival function prediction in cox.ph family. Code used expression
(8.8.5) in Klein and Moeschberger (2003), which is missing a term. Correct
expression is, e.g., (10) from Andersen, Weis Bentzon and Klein (1996)
Scandinavian Journal of Statistics.
* Added help file 'cox.pht' for Cox PH regression with time dependent
covariates.
* fix of potential seg fault in gdi.c:get_bSb if single smooth model
rank deficient (insufficient workspace allocated).
* gam.fit5 modified to step half if trial penalized likelihood is infinite.
* Fix so that bam works properly with drop.intercept=TRUE.
1.8-14
* bug fix to smoothCon that could generate NAs in model matrix when using bam
with numeric by variables. The problem was introduced as part of the
bam(...,discrete=TRUE) coding.
1.8-13
* Added help file ?one.se.rule on the `one standard error rule' for obtaining
smoother models.
* bam(...,discrete=TRUE) no longer complains about more coefficients than data.
* 's', 'te', 'ti' and 't2' modified to allow user to specify that the smooth
should pass through zero at a specified point. See ?identifiability.
* anova.gam modified to use more appropriate reference degrees of freedom
for multiple model call, where possible. Also fixed to allow multiple
formulae models and to use -2*logLik in place of `deviance' for
general.family models.
* offsets allowed with multinomial, ziplss and gaulss families.
* gevlss family implementing generalized extreme value location, scale and
shape models.
* Faster code used in 'uniquecombs'. Speeds up discretization step in
'bam(...,discrete=TRUE)'. Could still be improved for multi-column case.
* modification to 'smoothCon' to allow resetting of smooth supplied
constraints - enables fix of bug in bam handling of 't2' terms, where
parameterization of penalty and model matrix did not previously match
properly.
* clarification of `exclude' argument to predict.gam in help files.
* modification to 'plot.gam' etc, so that 'ylim' is no longer shifted by
'shift'.
* ylim and ... handling improved for 'fs' plot method (thanks Dave Miller)
* gam.check now recognises RStudio and plots appropriately.
* bam(...,sparse=TRUE) removed - not efficient, because of unavoidability
of dense off diagonal terms in X'X or equivalent. Deprecated since 1.8-5.
* tweak to initial.sp/g to avoid infinite loop in s.p. initialization, in
rather unusual circumstances. Thanks to Mark Bravington.
* bam and gam have `drop.intercept' argument to force the parametric terms not
to include a constant in their span, even when there are factor variables.
* Fix in Vb.corr (2nd order edf correction) for fixed smoothing parameter case.
* added 'all.vars1' to enable terms like s(x$y) in model formulae.
* modification to gam.fit4 to ignore 'start' if it is immediately worse than
'null.coef'.
* cSplineDes can now accept a 'derivs' argument.
* added drop.intercept handling for multiple formulae (mod by Matteo Fasiolo).
* 'gam.side' fix to avoid duplication of smooth model matrices to be tested
against, when checking for numerical degeneracy. Problem could occasionally
cause a failure (especially with bam), when the total matrix to be tested
against ended upo with more columns than rows.
* 4 occurances of as.name("model.frame") changed to quote(stats::model.frame)
* fix in predict.bamd discrete prediction code to be a bit more relaxed about
use of as.factor, etc in formulae.
* fix in predict.gam handling of 'na.action' to avoid trying to get type of
na.action from name of na.action function - this was fragile to nesting
and could cause predict.bam to fail in the presence of NA's.
* fix of gam.outer so that general families (e.g. cox.ph) can have all
their smoothing parameters supplied without then ignoring the penalties!
* fix in multiple formulae handling of fixed smoothing parameters.
* Fix of bug in zlim handling in vis.gam perspective plot with standard
errors. Thanks Petra Kuhnert.
* probit link added to 'jagam' (suggested by Kenneth Knoblauch).
* 'Sl.' routines revised to allow operation with non-linearly parameterized
smoothers.
* bug fix in Hessian computation in gam.fit5 - leading diagonal of Hessian of
log|Hp| could be wrong where Hp is penalized Hessian.
* better use of crossprod in gamlss.gH
1.8-12
** "bs" B-spline smoothing basis. B-splines with derivative based penalties
of various orders.
* 'gamm' now uses a fixed scale parameter in PQL estimation for Poisson and
binomial data via the `sigma' option in lmeControl.
* bam null deviance computation was wrong with prior weights (including
binomial other than binary case), and returned deviance was wrong for
non-binary binomial. Fixed (did not affect estimation).
* improvements to "bfgs" optimizer to better deal with `infinite' smoothing
parameters.
* changed scheme=3 in default 2-D plotting to grey scale version of
scheme=2.
* 'trichol' and 'bandchol' added for banded Cholesky decompositions, plus
'sdiag' functions added for extracting and setting matrix sub- and
super- diagonals.
* p-spline constructor and Predict.matrix.pspline.smooth now allow set
up of SCOP-spline monotonic smoothers, and derivatives of smooths. Not
used in modelling functions yet.
* s(...,bs="re") now allows known precision matrix structures to be defined
using the `xt' argument of 's' see ?smooth.construct.re.smooth.spec for
details and example.
* negbin() with a grid search for `theta' is no longer supported - use
'nb' instead.
* bug fix to bam aic computation with AR rho correction.
1.8-11
* bam(...,discrete=TRUE) can now handle matrix arguments to smooths (and hence
linear functional terms).
* bam(...,discrete=TRUE) bug fix in fixed sp handling.
* bam(...,discrete = TRUE) db.drho reparameterization fix, fixing nonsensical
edf2. Also bam edf2 limited to maximum of edf1.
* smoothCon rescaling of S changed to use efficient matrix norm in place of
relatively slow computation involving model matrix crossproduct.
* bam aic corrected for AR model if present.
* Added select=TRUE argument to 'bam'.
* Several discrete prediction fixes including improved thread safety.
* bam/gam name gcv.ubre field by "method".
* gam.side modified so that if a smooth has 'side.constrain==FALSE' it is
neither constrained, nor used in the computation of constraints for other
terms (the latter part being new). Very limited impact!
* No longer checks if SUPPORT_OPENMP defined in Rconfig.h, but only if _OPENMP
defined. No change in actual behaviour.
1.8-10
** 'multinom' family implemented for multinomial logistic regression.
* predict.bam now defaults to using efficient discrete prediction methods
for models fit using discrete covariate methods (bam(...,discrete=TRUE)).
* with bam(...,discrete=TRUE) terms like s(a,b,bs="re") had wrong p-value
computation applied, as a result of being treated as tensor product terms.
Fixed.
* minor tweak to soap basis setup to avoid rounding error leading to 'approx'
occasionally producing NA's with fixed boundaries.
* misc.c:rwMatrix made thread safe (had been using R_chk_calloc, which isn't).
* some upgrading for 64bit addressing.
* uniquecombs now preserves contrasts on factors.
* variable summary tweak so that 1 column matrices in parametric model are
treated as regular numeric variables.
1.8-9
* C level fix in bam(...,discrete=TRUE) code. Some memory was mistakenly
allocated via 'calloc' rather than 'R_chk_calloc', but was then freed
via 'R_chk_free'. This could cause R to halt on some platforms.
1.8-8
** New "gp" smooth class (see ?gp.smooth) implemeting the Matern
covariance based Gaussian process model of Kamman and Wand (2003), and a
variety of other simple GP smoothers.
* some smooth plot methods now accept 'colors' and 'contour.col' argument to
set color palette in image plots and contour line colors.
* predict.gam and predict.bam now accept an 'exclude' argument allowing
terms (e.g. random effects) to be zeroed for prediction. For efficiency,
smooth terms not in 'terms' or in 'exclude' are no longer evaluated, and
are instead set to zero or not returned. See ?predict.gam.
* ocat saturated likelihood definition changed to zero, leading to better
comprability of deviance between model fits (thanks to Herwig Friedl).
* null.deviance calculation for extended families modified to make more sense
when `mu' is the mean of a latent variable, rather than the response itself.
* bam now returns standarized residuals 'std.rsd' if `rho!=0'.
* bam(...,discrete=TRUE) can now handle 'fs' terms.
* bam(...,discrete=TRUE) now accepts 'by' variables. Thanks to Zheyaun Li
for debugging on this.
* bam now works with drop.unused.levels == TRUE when random effects should
have more levels than those that exist in data. (Thanks Alec Leh)