Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add drop_singletons option to partial_out() #263

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Add drop_singletons option to partial_out() #263

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
8c9d038
Add drop_singletons option to partial_out()
droodman Apr 3, 2024
563c32d
Update .gitignore
droodman Apr 3, 2024
2f9dd38
Add dof_fes return value to partial_out()
droodman Apr 3, 2024
23688f0
Merge branch 'master' of https://github.com/droodman/FixedEffectModel…
droodman Apr 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions src/partial_out.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Partial out variables in a Dataframe
* `double_precision::Bool`: Should the demeaning operation use Float64 rather than Float32? Default to true.
* `tol::Real`: Tolerance
* `align::Bool`: Should the returned DataFrame align with the original DataFrame in case of missing values? Default to true.
* `drop_singletons::Bool=false`: Should singletons be dropped?

### Returns
* `::DataFrame`: a dataframe with as many columns as there are dependent variables and as many rows as the original dataframe.
Expand Down Expand Up @@ -40,7 +41,8 @@ function partial_out(
@nospecialize(method::Symbol = :cpu),
@nospecialize(double_precision::Bool = true),
@nospecialize(tol::Real = double_precision ? 1e-8 : 1e-6),
@nospecialize(align = true))
@nospecialize(align = true),
@nospecialize(drop_singletons = false))

if (ConstantTerm(0) ∉ eachterm(f.rhs)) & (ConstantTerm(1) ∉ eachterm(f.rhs))
f = FormulaTerm(f.lhs, tuple(ConstantTerm(1), eachterm(f.rhs)...))
Expand All @@ -67,6 +69,7 @@ function partial_out(
fes, ids, ids_fes = parse_fixedeffect(df, formula_fes)
has_fes = !isempty(fes)

drop_singletons && drop_singletons!(esample, fes, Threads.nthreads())

nobs = sum(esample)
(nobs > 0) || throw("sample is empty")
Expand Down Expand Up @@ -135,6 +138,8 @@ function partial_out(
residuals .= residuals .+ m
end

dof_fes = mapreduce(nunique, +, fes, init=0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would find sum(nunique(fe) for fes) easier to read


# Return a dataframe
out = DataFrame()
j = 0
Expand All @@ -148,5 +153,5 @@ function partial_out(
out[!, Symbol(y)] = residuals[:, j]
end
end
return out, esample, iterations, convergeds
return out, esample, iterations, convergeds, dof_fes
end
Loading