-
-
Notifications
You must be signed in to change notification settings - Fork 686
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support Linux? #332
Comments
Tying this to the windows support issue is where the dart_vlc plugin from @alexmercerind has been discussed to support windows and it now has suport for linux too. |
Since I personally use Linux, I may personally give this a try (no ETA, though, since I'm also working on several other issues - if someone wants to jump in before me, advertise your intent both here and on the windows support issue so that we don't duplicate effort.) |
I'm thinking about using ALSA library with FFI to support audio on Linux. |
@GZGavinZhao no need to think about it, I've got a package already with simple playback working, PR contributions most welcome. Sorry @ryanheise I forgot to report back here when I started working on it. If you'd like something more cross-platform, I can recommend the libao package which I've been happily using recently. |
@maks That's awesome! I have thought about Also, I'm a little bit worried about |
Options options options! There is no problem having multiple implementations in a federated plugin framework. In #103 we had come up with a naming convention like this:
For example:
From the libao page it looks like there are multiple targets for linux, but we could still fit them into the above naming framework by treating |
My requirements for are for realtime audio synthesis, sample-playback and filter application (delay, reveb, etc) which I think are very different from just_audio's focus on media file/stream playback, compressed format support, metadata management etc, so I'm using Alsa and Libao directly via my packages, but like I said I'm happy to accept PRs that add whatever functionality would be needed to for it to be used by this package for linux support. But again given the rich feature set that just_audio already supports, I think you may find that using something like MiniAudio would work better, given its support of many of those features, even if you don't need it's multiplatform support, even though on Linux Pulseaudio is now the most widely used audio server, direct Alsa use is much less common and Jack is widely used for music-production low-latency applications. |
@ryanheise yes that naming sounds good, let a thousand flowers (platform implementations) bloom! 🙂 👍🏻 |
Was able to get |
@GZGavinZhao that's cool, but what's easiest is to map dart_vlc to just_audio's platform channel interface. You guys don't need to worry about playback backends (or anything native), dart_vlc is being maintained by me & a lot of time has been already spent making it stable & fixing issues reported by the users. dart_vlc is also part of awesome-vlc page from one of libVLC community members here. You can just add dart_vlc as a dependency to just_audio & use it (submit a PR to this repo). I'm also using FFI in this project now after migration from platform channels (until that time I couldn't figure out async callbacks). Just using FFI wasn't enough, I have also added event callbacks for position, video frames etc. through NativePorts of Dart VM. You just need to add Dart mapping to dart_vlc for just_audio. If you are not statisfied with dart_vlc's interface, you can use baremetal C-like dart_vlc_ffi package that dart_vlc internally uses (however then you have to setup your own cmake script in just_audio to build Speaking of miniaudio, alsa or libao... Those libraries are really really low level, even writing network streaming from scratch will give us a lot of pain. libVLC already offers a nice interface & dart_vlc brings its most features to Dart. @megamegax spent bit of time earlier trying this. But hasn't done much after that. https://github.com/megamegax/just_audio And, let me know in the issues of dart_vlc if you guys face any kind of problem or issue. I'll be fixing them as best as I can. Windows support for just_audio: #103 (also can be targetted by same work) Thanks. |
For Windows, I have even better idea to use C++/winRT APIs themselves (Windows 10+ specific however, I don't know what to say to people who use < 10) in future. Why deal with libao, libao buffers, decoding & all... (So, I'm working on an another separate side side side project to expose those APIs in a DLL, so that Dart VM can use it). miniaudio is like a dead-end at this point unless one is really really geek to build things like network streaming, better format support on top of it. Speaking of miniaudio, I already worked on it (possibly world's first) to play audio on Flutter desktop, flutter_audio_desktop, but dart_vlc is just better. |
@alexmercerind Wow thanks for the suggestions and resources! I'm checking it out now. One concern I do have is about external dependencies (which was why I first tried |
@GZGavinZhao on Windows, I provide the libvlc source directly within the plugin & all the shared libraries are automatically copied with CMake (I have everything ready already, don't worry about that). On debian, one must have vlc & libvlc-dev installed with apt (I believe depending on distro's repos is best bet, building libvlc can take ages). Someone has also added info to build it on redhat based distros. Not sure about arch however (although people have built it successfully on manjaro). |
@alexmercerind Understood. Thanks for the detailed explanation! |
@GZGavinZhao great work getting miniaudio working, do you have a repo with that FFI binding? I started doing a package for FFI binding but didn't get very far before switching to Alsa and then libao. I'm not sure why @alexmercerind you said "FFI isn't enough"? You just need to call sync C APIs like libao in a isolate to ensure you don't block the main isolate. I've done a simple implementation of that recently for my project and I'll write up an article on how to do that soon. And yes with respect to libvlc you will need to ship it as a shared lib in your Flutter app to your users and be aware that while libvlc is LGPL licenced some modules maybe be GPL so that would mean your app then needs to be GPL too. It's all really dependent on what you need. I chose to use Alsa and libao (miniaudio on my to-do list too) as I do need low level, sample level API access. If you instead only want a higher level API with features like streaming or compressed formats then something like libvlc is probably better choice which it might be given that just _audio seems to be more focused on those use cases. |
Yes, FFI is not enough unless they fix this: https://dartbug.com/37022. Its not about sending messages to C/C++ from Dart, but its also about being able to receive messages from C/C++ to Dart. miniaudio in comparison to libvlc or winRT APIs is very basic, there isn't much like event callbacks. In just_audio, there are many event streams (as Dart has those great things) e.g. for position, playlist & other events etc. etc. (github.com/ryanheise/just_audio/blob/e312da2a210ac7b5c0c4ba85cc59c2c832cff52b/just_audio/lib/just_audio.dart#L301-L404) See how I had to use NativePorts to send back position, video frame buffers, playlist events etc. etc. asynchronously. https://github.com/alexmercerind/dart_vlc/blob/master/ffi/native/eventmanager.hpp because FFI wasn't enough. Now one might say why use Stream or async callbacks, just leave it... We can't because that's how a Dart library is made, all developers are familiar with Stream subscriptions & its the easiest way to update the UI. Thus, for notifying about events being happened asynchronously, its necessary to send updates back to Dart. More on what problem is here in this comment: #103 (comment). In C++/winRT aswell, one can register for event callbacks by passing a lambda (and that will get fired when certain thing happens), but using that in FFI will certainly result in a crash or use NativePorts.
libVLC can be compiled with only LGPL modules enabled. And difference will be unnoticeable. If one doesn't even wanna compile, they can just delete the shared libraries.
That's what I also believe. For just_audio, general high-level interface to audio playback is required with features like network streaming, good format support, event callbacks etc. and its just hard to do in miniaudio or from bare alsa/libao & if one decides to do it themselves (it might be another media player from scratch...), it will be far from just a "binding" or a "plugin". |
Thats correct, because we are talking about FFI not platform channels, so we are not sending any messages, instead with FFI we are making function calls. I don't know what API libvlc exposes, but Alsa and libao have synchronous APIs so theres no need for async callbacks from native code into Dart. As I said you simply need to use a seperate Isolate that then exposes an event stream to ihe main Isolate. You don't give up on using Streams, you simply use standard Send/Recieve ports that Dart provides for all interisolate comms. The main isolate can then expose whatever streams are required using messages coming in from second isolates receive port. Here is a gist with code extracted from one of my recent projects that demonstrates the basic of this approach. My code is using sample based playback with libao, so its trivial to have this code report a playback stream in the main Isolate. |
@maks Thanks! I wouldn't call it a proper binding though, because technically I didn't wrap the @ryanheise Just found a problem: because
Any work-around for that? Like creating a temporary empty implementation? |
@GZGavinZhao a vaguely remember that audio session type functionality is available via DBus on Ubuntu and many other distros and there is already a DBus package available written by one of the Ubuntu staff developers. |
Hmm, I thought I had already implemented audio_session as a noop on other platforms. That is something I should fix: |
OK, this is now implemented in the git version of audio_session but I'll hold off doing a release until we can test whether this solves the issue. |
Hi @megamegax I was just taking a look at your code and it looks like quite a clever approach! Basically, to start with the existing web implementation as a base, since it already implements all of the backend logic, and then just substitute in the vlc player instead of the HTML5 audio element. Of course eventually we could reimplement the backend logic using vlc features like playlist, although that could be a substantial amount of work. Are you interested in continuing on your project and/or collaborating? Or would it be best to work on it as a sub-project of the main just_audio repo? The idea of the federated plugin model is that in theory anyone can create an independent platform implementation and publish it themselves on pub.dev without my needing to be involved, but if it helps to get this thing off the ground, I don't mind having a more central effort. By the way, if you are willing to keep working on it, I'd suggest opening up the GitHub issues page to invite collaborators to help submit bugs, etc. as well as get the license and copyright notice in place. Then I could also link from my project to your page and give it more attention. Either way is workable to me. |
@ryanheise Thanks for the quick help! Progress is unfortunately blocked because of issue flutter/flutter#52267, which basically says that there's no way of to creating a Dart implementation of a platform interface except for web. The problem is that if we use They did provide one way to manually force the use of pure Dart plugins (as shown in this line here taken from the first-party plugin |
@GZGavinZhao Yes, I was coincidentally looking at path_provider last night for the same reason. So in the VLC case, something like this is now possible: flutter:
plugin:
implements: just_audio
platforms:
linux:
dartPluginClass: JustAudioVlcPlugin
pluginClass: none And then inside the Dart file we can have: class JustAudioVlcPlugin extends JustAudioPlatform {
final Map<String, JustAudioVlcPlayer> players = {};
/// The entrypoint called by the generated plugin registrant.
static void registerWith() {
JustAudioPlatform.instance = JustAudioVlcPlugin();
}
...
} But no it won't become endorsed unless the main frontend plugin's pubspec file declares it so. Platform implementions can't endorse themselves. By the way, it is still technically possible to get a Dart platform implementation working without the above feature, as I have used a workaround in |
@ryanheise I tried to use this approach with the pubspec file but it didn't work. :( In case you're wondering, the reason I added an extra "linux" after the plugin name and class is because I want to call some native code to do native volume controls and notifications, and this can help differentiate from the Windows implementation. |
Just a quick comment, on the plugin name, but if the plugin is written entirely in Dart and you move all of the native code into a separate dependency, and if consequently the Windows and Linux Dart implementations are largely the same, you can also just write if/else statements depending on the platform. |
I just tried the path_provider example on linux and it works. I then tried un-endorsing the linux implementation and manually added it as a dependency to the example, and it still worked. It's still a mystery to me what could be the difference between that and your project, but maybe it's a starting point for investigation? |
Nice work! I will update the README to link to these. Just a note on the feature sets, reading from a byte stream, and request headers, should (I would expect) be supported for free because they are handled by the plugin's front-end by setting up a proxy, and then passing the proxy URL to the platform implementation. As long as the platform implementation handles URLs, then it will (or should) implicitly support these other two features. There is a similar situation where if the platform supports loading audio from a local file, you will also get the ability to play from assets for free. |
@bleonard252 I am about to publish an updated README pointing to your just_audio_mpv plugin. I was thinking it might be helpful if you also update your README to contain installation instructions, specifically about installing the |
I'm about to go to sleep but I'll do that in the morning.
|
mpv is already available in debian & ubuntu repositories & of course The AUR has it already. Both flatpak & snap also have it available in their repos |
the |
👍 looks good. I think that link to the MPV installation page covers all bases. I forgot this is actually a Linux and Windows implementation, but I suppose that is similarly handled by that link to the MPV installation page? |
Just wondering, why does it say to get |
@ryanheise Well, MPV works on Windows/MacOS too but the |
I see, in that case I think it will be necessary to fix this in the just_audio_mpv plugin: plugin:
implements: just_audio_platform_interface
platforms:
linux:
dartPluginClass: JustAudioMpv
fileName: just_audio_mpv.dart
windows:
dartPluginClass: JustAudioMpv
fileName: just_audio_mpv.dart The windows declaration above can be deleted and perhaps re-added if mpv_dart some day adds support via win32. Then what I can do for now is update the just_audio example so that it depends on both just_audio_mpv and just_audio windows, where each one declares support for a separate platform. Right now, I'm not sure if there is a way for an app to selectively use just_audio_mpv for Linux and just_audio_windows for Windows, since then there are two plugins both declaring that they provide the Windows implementation. I don't know, maybe whichever one's declared second ends up overriding the first. There should probably be a feature request to the Flutter team to provide explicit options to select the plugin that's capable of supporting multiple platforms only for a specific platform. |
The claim of Windows support has been removed. also the latest push to Pub now includes (frankly untested) fedora instructions since I forgot to with 0.1.2 earlier |
About this, I noticed @alexmercerind created I haven't currently tested on Windows, but since I'm not doing platform-specific stuff I'm guessing it'd work. |
package:media_kit offers both video & audio playback on all platforms. You can efficiently select native libraries for minimal bundle size. The stability, performance & codec-support is solid (there are many unit-tests). Since it uses FFI, the feature implemented for one platform automatically works on all platforms (only video rendering is platform specific); and there's a lot that libmpv offers. A good idea may be to let dependents select media_kit_libs_*** instead. There's no hard dependency on native libraries. Similar to package:just_audio_media_kit, project provides package:video_player_media_kit for zero-cost implementation / migration to package:video_player users. |
Nice, @Pato05 (and @alexmercerind )! Regarding request headers support, as you may or may not know, this is currently handled on the Dart side through just_audio's proxy, but I am currently working on changing this so that if the platform is able to support headers directly, the proxy won't be used. So the current code you have to pass through the headers will currently get overridden by the proxy, although in the upcoming change it will be exposed. @alexmercerind , would you happen to know if mpv has any API that would facilitate playing clips? For example, the ability for playback to automatically stop at a precise timestamp? |
That makes sense... I've put the hard dependencies because there wouldn't be any reason to use the plugin other than to also support Linux and Windows since By the way, great package! |
Oh, I didn't notice that while looking at |
I'd personally be interested in using |
@jmshrv Beware that the native library is licensed under GPL which is feasible for distribution of open source apps on the desktop, but incompatible with the Play Store an App Store even if those apps are open source. |
Yes, there is. However not exposed in the package's API. We do provide an API to tinker with internal libmpv properties. There's a lot that libmpv offers & it's very easy to implement & test with FFI. Refer to --start & --end: https://mpv.io/manual/stable/
package:media_kit is fully compatible with commercial usage. We compile our own builds for reduced bundle size & LGPL compatibility, you can find relevant repositories here. |
Thanks for the insight into mpv's capabilities, it's good to know that clipping is theoretically possible. I wasn't aware the native lib was LGPL rather than GPL although unfortunately even the LGPL is incompatible with the play store and app store, because it requires giving users the freedom to be able to re-link your app to a different version of the LGPL library and then run it which is not really feasible in a locked down system that users get through the app stores. This issue also affects ffmpeg. There is no issue with desktop apps, though. |
That would still be possible by manually calling the |
I don't happen to be a lawyer. Both (lib)mpv & (lib)VLC used to be only GPL. The re-licensing has been largely motivated with the aim to allow their usage in proprietary/commercial applications: mpv-player/mpv#2033 There are many applications that are distributed on Google Play or AppStore, which include dynamically-linked LGPL dependencies (even libmpv or FFmpeg). If someone * really * wishes to replace the native libraries bundled in an APK etc., I believe that is also possible with some repackaging. Regardless, one can't really do anything if operating system itself restricts users' actions. Linking dynamically as a .dll (Windows)/ .so (Android, GNU/Linux)/ .dylib (macOS & iOS) is enough to comply with LGPL. It's more like the code-push situation e.g.
I'm unsubscribing to this thread, so I won't be receiving any notifications. It's getting bit noisy for me. |
I don't happen to be a lawyer either.
The problem @ryanheise is referring to is nothing to do with the OS but the policies, app developer licenses and terms of service of the App Store and Google Play. Those are legal agreements between an app developer, users and Apple/Google, just because they restrict their users freedoms, hence the long standing issue with distributing GPL'd software via the App store. @alexmercerind App store users and app developers are not allowed to comply with the provisions of the GPL hence its not up to you to say "if they really wished to". I also have no idea why you referenced shorebird, that is a specifically different set of provisions of the Apple Developer Program License Agreement (section 3.2.2) that they are referring to due to its prohibition on executing downloaded code and really has nothing to do with the issue at hand here. |
I don't happen to be a lawyer.
I don't understand on what basis (without source), you say: According to LGPL (to my understanding), the end-users must have an option to replace/update the linked LGPL code. The closest that one can get (is expected & followed) to allowing this on any platform (not just Android & iOS) is linking the LGPL code dynamically. If users * really * wish to dive in & replace the default LGPL shared libraries, they can (& there are methods that allow, of-course not "official" by Google Play or AppStore). It must be kept in mind that Google Play or AppStore are just a way to distribute applications. You could even argue that using dynamically linked LGPL libraries in an application distributed on Microsoft Store is not legal since it bundles the application differently.
I referenced it at the end of my comment, because it has a similar situation & no clear answer. The legal process is entirely different. At many times, it has been seen that things boil down to how good your legal team or lawyer is. From what I've known & told so far, it is fine. I don't have time to closely read every license & agreement to convince anyone in support/against based on any clause or section that I find. Like I said in my comment earlier, there are thousands of applications available on Google Play & AppStore which include dynamically-linked LGPL dependencies in form of shared libraries. Both libmpv & libVLC were re-licensed with primary aim of allowing their usage in commercial/proprietary applications, I don't think it offends them. You can read the linked mpv issue to understand their motivation & why exactly LGPL (& not MIT, Apache-2.0 etc.). FYI: any improvements that are made to the package are still open-source (either to package or mpv directly). You are free to make your own decisions & hold accountability. Use package:media_kit or don't. We had this discussion a lot time ago internally while starting to work on project. |
@alexmercerind and as I said, I don't happen to be a lawyer either.
it's nothing to do with using loose language like not being "official", its not some sort of grey area, its about breaking the terms of service of the stores.
Exactly, that I 100% agree with you on, as a developer doing due diligence for development of apps for a commercial client for to be deployed via App Store or Google play, I can easily recommend the use of libraries under an MIT or BSD license, I could not do so for LGPL licenses ones, the choices you make are of course up to you. |
(IANAL)
Indeed there are many apps on the app stores that are not compliant with open source licenses. A 2011 study found 71% of apps using open source libraries to be non-compliant with their licenses. A more recent 2017 study found a much smaller percent although still 40K apps. The fact that many people do it doesn't itself make it legal, so it would probably be best advised for each app developer to consult their own lawyer. One more interesting link: Google themselves warn about using LGPL on Android, stating:
For further reading, here is a StackOverflow post that gives a fairly decent summary of the issues. There was also an ffmpeg thread I can't find right now where the issue was discussed, and even if the authors would agree in principle apps "should" be allowed to use the software on iOS/Android, or at least that usage is in the spirit of what they might have originally intended (before the app stores came about), now that the app stores are here and there is this license incompatibility, it was basically concluded that ffmpeg has so many different authors over so many years that it would be next to impossible to get everybody's agreement to change the license to something that would be compatible with the app stores. Now, it may technically be possible in some situations to provide instructions to the user on how to re-link your app and rebuild it and then install it, perhaps more feasible on Android than iOS, and a few years ago I went down that path trying of trying to be fully compliant by doing this, but in the end I gave up. @alexmercerind you are completely right that the software can still exist, and it is simply up to the app developer whether to use it or not, after coming to their own legal determination (ideally in consultation with a lawyer, of course). |
Informally speaking, I draw my line at GPL ;) Thanks for sharing some interesting metrics @ ryanheise. I have shared the mpv thread regarding their LGPL re-license. I personally believe that "the spirit of what they might have originally intended" is preserved. My legal talk quota is reached for this month. 😄 |
That was.. an interesting discussion to say the least. On the other hand, are there any concerns with distributing app bundles on desktop? I know that the linux version requires mpv installed on the system to work. I wonder whether it's the same on Windows though. About Android, shouldn't mpv-android on the Play Store have the same issue? It seems to based on libmpv, just like media_kit |
I've said it before & I'll say it again there is no issue. There are thousands applications (big & small) available on AppStore & Google Play that use dynamically-linked LGPL dependencies. For instance, there's ijkplayer, specifically for Android & iOS from Bilibili, which is LGPL. Just link dynamically to external LGPL code & you're good. You aren't offending anyone. "Re-linking" is often as simple as deleting old file & adding new one.
That's how GNU/Linux works. If you see closely, you'll see we don't provide own libmpv builds for GNU/Linux. On other platforms, there is zero configuration required outside of adding packages to your pubspec.yaml. I have provided direct download links to demo applications in the README for each platform for you to evalulate. |
In the case of mpv-android, it is an open source app, and so it would seem to trivially satisfy Clause 6a of the LGPL. In a closed source app, the case on Desktop is also quite trivial by Clause 6b since you dynamically linked libraries to allow the library to be stored in a separate file and you could then easily just replace the file without rebuilding the app. On Android, a user with a typical Android phone won't be able to do that, so you would need to provide them with a way to rebuild your app with the alternative library based on your object code rather than your source code. There are some cases in which this is possible, and some cases in which this is not possible. But in cases where it is possible, it is still a rather complicated process. You should at least demonstrate that it is possible, and then provide your users with a way to do that. In my opinion, it's a rather impractical clause in the context of mobile apps, and if I were in your shoes, I would first try to fully comply with the license, then if that proved too difficult, I would then try to either persuade the authors to remove the problematic clause from the license, or try a substitute with a more acceptable license. |
Is your feature request related to a problem? Please describe.
At the moment it looks like support for window is in progress, if Linux can also be supported then great~~
Describe the solution you'd like
n.a
Describe alternatives you've considered
n.a
Additional context
n.a
The text was updated successfully, but these errors were encountered: