Doesn't always recreate types on changed SQL source when using SqlFile #335
Comments
cmeeren
changed the title
|
I don't think there is an easy way around from the TP standpoint and I agree it is annoying. My current approach is to have the file containing the provider instanciations be generated by a script, that script will regenerate this file if content of .sql has changed or the file is out of date. It also allows me to embed some metadata in my .sql files that has an impact on how the generated F# code looks like. Another potential workaround is to have the .sql file be part of your project in some degree, so msbuild would decide to recompile since a file has been touched since last valid compile. I have not explored this. Another one would be a I'd rather keep the compilation model behave the same, people get the same issue with editing files pointed in .resx files for example, only a clean or rebuild address this. If there are ideas how this could be implemented, I'm eager to learn about them. |
|
Thanks for the reply. Unfortunately, complicating our process with external tools is not a viable solution for us.
They are indeed part of the project (primarily for easy access from VS). I don't know how to tell MSBuild to recompile, though. I figured it was supposed to happen automatically. Do you have any pointers? |
|
Regardless of MSBuild though: When using SqlFile, I would expect it to pick up on file changes automatically, making the changes immediately visible. Is this unrealistic? Don't type providers normally work that way? |
no, but it is optimistic and I guess requires a lots of gaps to be bridged. I don't have deep enough knowledge about type provider infrastructure if this is the place changes would need to happen.
If you are able to figure a better workflow please share it. I believe looking more into msbuild and how it tracks project resources is the best short term and integrated solution in your context and could be part of recommendations in the documentaiton. |
|
Thanks! Yes, I agree that the documentation should be updated with a warning about forcing rebuild to make absolutely sure, and if possible also recommendations about using MSBuild to automatically rebuild when SQL files have changed. |
|
Tried building master using
The code that produces this is module Foo
open FSharp.Data
type Foo =
SqlCommandProvider<
const(SqlFile<"""Test.sql""">.Text),
@"Data Source=localhost\SQLEXPRESS;Trusted_Connection=Yes;"> |
|
@cmeeren could you give some precision about the IDE / dotnet sdk version that are involved in your case? I've been checking the changes in VS2019, but the error looks like this is running on netstandard or something like that. Right now, the design time doesn't support netstandard, this is tracked in issue @matthid just opened: #337. For now, either mono or netframework are needed for the design time / IDE, despite code compiled against the provider can ship / deploy on netstandard. |
|
Yes, this was a .NET Standard library. I'll try again using the full framework. |
|
I can confirm that the changes work as you describe: Any edit to the .fs file (even without saving) will seemingly invalidate the VS cache. This makes it easier to trigger the update in the IDE, which I appreciate. However, I just want to be clear that this is not really a workaround or fix for this particular issue. This issue centers on the build experience, not the IDE experience (sorry if that wasn't clear). It is still the case that if a project is built, and then an SQL file edited, a new build (normal build, not rebuild) will still skip the project because VS believes it is already built and up-to-date. This happens even if you make an edit and then undo it, regenerating the types in the IDE; the IDE may then say that the SQL is invalid, but since no files have changed, the project is still skipped when building. Conversely, if you simply re-save the .fs file without modifying it, the IDE does not update, but the project is re-built and errors picked up there. For this issue to be considered fixed, any edit to the SQL file must cause a re-compilation (and preferably also a refresh in the IDE). We can't and shouldn't depend on devs remembering to make a change to relevant .fs files whenever SQL files have been modified. Again, though, I appreciate the simplicity of refreshing VS - it was a pain to re-load the project, and that particular issue is now at least remedied, if not fixed. As for a potential solution: Would it be possible to use FileSystemWatcher for this? |
|
Top result for "type provider watch file"; might be relevant:
(emphasis mine) |
I noticed the same problem but that it is only problem in development phase for myself. Release will always be the full rebuild. If someone can propose some way to find out what fs files should be recompiled when sql file is changed then I'm more than happy to try to fix that problem also. Another option I was thinking is that is it possible that this recompile problem would go away if these SqlFile types would generative types instead of erased types. I'm not too familiar with type providers but if I understood the difference right then these generative types would be in own assembly and when that is changed then VS should be able to understand that it should recompile files that depend on types in that assembly. |
Doing the change for the library to be 100% generative is not a small task, I'm not familiar enough with quotation stuff happening beside observations on compiled code and adjusting some of those quotations. If we could compile those to IL it could remove one barrier, I still don't think this would solve the matter of pressing f5 after saving a .sql, what is needed is a couple of msbuild guidelines in the documentation requiring to add the .sql files to the project in some fashion so they get added as a dependency to rebuild the assembly. Involving the compiler, the TP SDK and/or type provider implementation feels much more complicated to me. Few guidelines for end users:
|
It is certainly best practice. But unfortunately many "older" companies are caught in a quagmire of legacy systems where MSDeploy, aka "Right click project in VS and click Publish" - or heck, even "Build project in Release mode and copy to deploy" - is the standard (and perhaps currently the only viable) method of deployment, with a giant mono-repo, no CI/CD pipelines set up and no resources (or expertise, for that matter) to prioritize it. (If it sounds like I'm speaking out of personal experience, that's no coincidence. ;) ) |
|
Hmm, strange. I found a project setting that I thought would help: <UpToDateCheckInput Include="**\*.sql" />What happens is that the project is seemingly re-built (if I interpret the VS output window correctly) if the SQL files are edited, but unless any F# source files are edited too, the TP does not pick up any changes. I tested this by compiling a project, editing an SQL file to be invalid, and then compiling the project again. The project compiles successfully (which it shouldn't), whereas if I edit an F# source file too, compilation fails due to a TP error (correct behaviour). (This was tested using the currently released SqlClient, but I don't think that matters much, because the workaround posted previously in this thread still requires an F# source file edit and just impacts the IDE experience, not the build experience.) |
|
Will #332 be released soon? Would be great to have it. :) |
|
A workaround for the problem in my previous comment is to place the following in the project file (in addition to the <Target Name="TouchProjectFileIfSqlChanged" BeforeTargets="BeforeBuild" Inputs="**\*.sql" Outputs="$(MSBuildProjectFile)">
<Message Text="SQL files changed. Changing project file modification time to force recompilation." Importance="High" />
<Exec Command="PowerShell -NoProfile -ExecutionPolicy Bypass -Command "(dir $(MSBuildProjectFile)).LastWriteTime = Get-Date"" />
</Target>This is part of #357. |
|
The tooling has changed since then, and there seems to be more agressive caching in Visual Studio at least. The original issue was solved via #342 |
I have several times experienced SqlClient/SqlFile not updating generated types when source SQL files change. This is annoying in the IDE (where this behavior is fairly consistent), but is worse when a solution build does not generate types for the current SQL files, but instead for old SQL files.
Just now, I even experienced bugs in production (published through VS, with a different build config) due to SqlClient not picking up the file changes (even when the solution config changed).
This seems like a fairly serious issue (provided it actually is a bug with SqlClient).
I have no minimal repro because this doesn't happen all the time.
A re-build always fixes the problem. But for obvious reasons I don't want to have to remember to always manually re-build before publishing.
In case it's relevant, I use SqlClient like this:
The text was updated successfully, but these errors were encountered: