Tempest Roadmap

[K-vanc]

Hi everyone. In last couple of months, I received some really good feature requests/advises from you which some of them was already in my to-do list and others were worth to do it. So I am really glad to see as we are thinking same and expecting same things to make Tempest the best grabber ever which it already is for me :) So, I want to mention about my roadmap for Tempest and you can also comment/add your requests if any;

• User Logo Option (Done as of rv1.4.3)
  I am working on a new command for some time to let users add their own logo urls for each channel. For unlocked siteconfigs, this can be already done but for encrypted siteconfigs it was not possible. With this option, you can add or change logo url for each channel on all siteconfigs

• How to Use Wikipages
  With latest patch, all siteconfigs are working flawlessly even in android servers. To take advantage of this, I will prepare Wikipages with screenshots for setup and usage on Android(KSWeb) , Windows(Xammp) and Linux(probably Nginx). This guides will help much for unexperinced php users.

• CLI Channel Creation (Done as of rv1.4.3)
  Tempest currently can make folder structure and can grab in CLI mode without any issue but for channel creation GUI is needed. I will code it allow channel creation also in CLI mode.

• Siteconfig Updater (Done as of rv1.5.0)
  I am planning to add a small script to download all siteconfigs(as updated) in zip. I started to use github with Tempest and was thinking when I download a release version of project as zip, i will get everything as latest, but I found out(in a hard way) that i am downloading project as the time of release.

• Docker Release (Done as of rv1.6.0.0)
  I believe this is the correct time for creating an official server + Tempest container after completing above steps

• Some improvements on Debug into File (Done as of rv1.5.3)
  This is just for torturing myself

• Append/Overwrite option for page grabbers (Cancelled since I am feeling too lazy for it)
  This is also for making my life easier during siteconfig making.

[Delitants]

Looks promising. But how to automate generation (refresh every n days) via cron after config is created?

[K-vanc]

For scheduled operations, I recommend enabling Fusion mode and setting level to 0 for all from Tempest Configurator. This will review grabbed shows and update changes on repeated operations. Also will restore previously grabbed show data in case on failure on new run. After that cli string and parameters are explained in page 49 of TempestWIKI.pdf. You can decide with options you need as parameter then add command line string directly into cron or prepare a .sh file, insert all commands to sh file and set .sh file to run by cron with your defined time interval. An example from my own sh file;

#!/bin/bash
sudo php /var/www/html/tempest.php engine=Generate createxmlgz=on createinvxmlgz=on

[ghost]

Is there any way to automate epg generation and upload that's file in dropbox (any other platform) in Windows? I'm currently running this in Xampp with CLI Mode
cd c:/xampp/htdocs
php tempest.php engine=Generate tempconfig=dbmtech.config.xml createinvxmlgz=on

how to automate this command everyday in Windows 11?

[K-vanc]

Create a .bat file such as epg.bat on your desktop, add your 2 lines into file and save. Now, press windows key and search for "Task Scheduler" in your pc. set the date/time options to run epg.bat based on your needs. For more info, you can google windows scheduler. Below example content not belongs to me;

https://www.thewindowsclub.com/how-to-schedule-batch-file-run-automatically-windows-7

For how to upload any drive etc. You may follow similar way since it is not my business. I am using rclone on my rpi to upload google drive but this part is out of my and Tempest's scope.

[ghost]

Ok! Thanks, I will try it.

[Delitants]

And how do I suppose to know which XML channel is in? Why there is no basic search by channel name?

[K-vanc]

And how do I suppose to know which XML channel is in? Why there is no basic search by channel name?

What this means by which xml channel in?
What this means by basic search by channel name?

[Delitants]

And how do I suppose to know which XML channel is in? Why there is no basic search by channel name?

What this means by which xml channel in? What this means by basic search by channel name?

Like, I open configurator, I can select xml source, and it lists all the channels in it. I don’t initially know which source contains my channel. I have to either do an ugly recursive grep across all site configs or click each one one by one in GUI and perform a search on the page by browser.
It’s just time consuming. Here is an example https://iptv-org.github.io/ searching channels and seeing the source for it. In your case I need to select source first and guess where the channel possibly can be.

[K-vanc]

Do you have any idea about how your example working? it is website hosted on a remote server which reading results from database files(there are ready several json files) with several javascript, jquery and maybe SQL database requests. Tempest is a single php script running on your local server which using basic html and very little javascript features. Configurator developed to minimize copy/paste and typing mistakes which causing syntax errors when directly working on configuration file and it may help people to save multiple selections safely. It supports basic alphanumeric search by pressing keyboard as limited and allowed by html. Same basic alphanumeric search also applicable for siteconfig names. For much more better search, you can use traditional NON-UGLY direct copy-paste method from xml to xml via any text editor or you can use some other NON-UGLY cli solutions as you got used to use(I dont know how you were handling this on them) but I cannot guess which channel you need and which siteconfig has it. Tempest is an epg generator, not a ready-epg hosting service

Anyway, i will have a look if i can add a quick search bar inside select box without breaking or recoding all structure

[Toei79]

I use rcclone too, honestly i just share with my family not about use for bussiness. my sister , my brother and my mother.

so far no good you need use dropbox premium.

im been using google drive but its not working. even for a playlist its get issues.

sometimes depends the site it gets the epg automatically without need to do manually, its just one time thing and its good to go. the plus all in one place.

[K-vanc]

Hi @Toei79 . I cannot help on other software issues but probably it is related with your setup. I am using rclone as part of my Tempest run .sh file in crontab for years and it is synchronizing generated xml file with my google drive folder for more than 2 years without any touch. It shall be uploaded with "sync" command otherwise download link will change on each update. After uploading, the file shall be enabled for sharing in google drive settings so my friends are just using static google drive download link that I gave them, by pasting it on their iptv player's epg location path. Just keep in your mind that there is a daily limit of sharing file then your link will be temporarily blocked for downloading. I didnt use any other cloud setup rclone so cannot say anything about dropbox.

[Toei79]

I was using dropbox it was good

i have my playlist there and gets blocked that can be issue.

I use tempest but i mix with other epgs that i did for radios kinda dummy thats fine for some reason no matter i change it don't detected some on the playlist and on the epg , im sure because locally its fine, but maybe can be google limit daily i never know about that , i only use myself at moment

[K-vanc]

Dear People of Tempest,

I need some suggestion regarding a possible feature. I saw some people using Tempest with port-forwarding or from hosting services so it is accessible from beyond firewall but that caused some security concerns (and also curiosity about how to do) on me then I wrote some block of code for additional login layer. It will shown as below;

Idea is, the users who needs login, will search in tempest.php file and for;

$username = '';
$password = '';

strings and hard code their credentials it into file such as;

$username = 'test';
$password = 'login123';

After saving tempest.php and reloading page, login layer will be activated. After clicking Logout, you will be back to login layer.

If you change hard coded username/password strings back into empty state, Tempest will deactivate login layer. Of course it is not a bullet-proof solution like a real website login but still will be an extra precaution with file renaming.

Anyway, I am not so sure about including that feature on my new release so wanted to get your opinions

[ghost]

will it effect automatic scheduled epg generation?

[Toei79]

nice thing, in my case its no necessary , i mean i share with my family barely knows about handle this stuff , myself take me like 6 months to pull the trigger, yea its not hard just im been like a year in linux stuff.

im been doing google drive but i dont get fully right at all honestly.

vpn tunnel? windergard i guess its called, thats safest? im been using asus feature instant guard on their routers, just start to check if i can install in other devices so it be useful. that can be safe? vpn tunnel?

[K-vanc]

will it effect automatic scheduled epg generation?

It will not effect CLI mode regardless of it is enabled or not. It will appear only on GUI mode(browser) requests. For users on localhost, it is completely useless. It may be useful only for port-forwarded or webhost users. These users may run Tempest on CLI mode via SSH connection and if you have SSH credentials, you can do anything you want so login layer will not have any purpose on CLI mode.

[K-vanc]

nice thing, in my case its no necessary , i mean i share with my family barely knows about handle this stuff , myself take me like 6 months to pull the trigger, yea its not hard just im been like a year in linux stuff.

im been doing google drive but i dont get fully right at all honestly.

vpn tunnel? windergard i guess its called, thats safest? im been using asus feature instant guard on their routers, just start to check if i can install in other devices so it be useful. that can be safe? vpn tunnel?

I think you mean Wireguard? It is similar like openvpn but using a new encryption method. As far as I know, it can be installed on many operating systems but I have no idea about security etc.

[Toei79]

yea that one, looks easier like other thing that you can install on your server or linux distro.

im think im going to take that path , its easier just install on remote device and mine, then they can use epg that i have here because its local for them.

asus have one but only supports cell phones and tablets. no android tv. the one from asus its easier one. no complex setup.

i did openvpn one time ago but barely understanding now i get a better logic of this , its a text file with some info there and here. wireguard its better for what i read.

[K-vanc]

Hi, actually it is not since all you need to do to set up a php server. I need some details such as;

What is your OS (Windows/Linux)?
Do you have a php server?
Do you know how to run a php file?

Please send your replies to my mail ( tempestepg@gmail.com ). I will reply you from there in Turkish since I am keeping this thread only for informative purpose about features/suggestions

[K-vanc]

New release coming soon, with self-update module..

[ghost]

Yeah it will be good. Appreciation for your efforts.. next waiting for docker release!

[K-vanc]

This actually will be a conceptual milestone for docker. I still not yet started to work on docker but reading when I have time. If my understanding is correct, they are changing the container for updates or you have to manually close the container and replace etc which is not so user-friendly. This will solve the dealing with containers problem. Otherwise, you can do the some by downloading and copy/pasting as you do now

[Toei79]

nice easier to update , never figure out docker i have a couple on my nas , i feel it complicated.

[K-vanc]

Ok guys, as promised some lonnnng time ago, managed to build and run first Tempest docker container yesterday and grabbed epg/created channel list on GUI and CLI modes. It will be on alpine linux build for minimum container size and with nginx + php8.2-fpm. Current container size is 100mb but it may be a little more or less depends on my final checks. Currently everything is running but for better multitasking, I need to tweak couple of things more on nginx and php since I notice some slow performance on detail paged siteconfigs compared with bare php performance.

In order to complete docker integration, there will be an Tempest update soon. With this update, Tempest will behave more like an application rather than script and as result generated epg and log files will be moved into tempest_config folder with their own folders as "epg" and "log".

This change is required for a smooth docker integration and users will be able access everything needed by just mounting tempest_config folder to any folder on their local host. With support of previous release for self-update, there will be no need to copy/paste tempest.php or rebuilding container.

Since i just learned basics of docker, i want to test more and be sure about everything before releasing it but this is just a advance notice for people who are waiting for docker that there is light now at the end of tunnel and notice for normal users regarding folder structure change on next update.

[ghost]

Great News 👍

[K-vanc]

first photo of application;

Around 90mb container memory usage

[naijaping]

Great Job Bro

[K-vanc]

And finally, docker release is available on;

https://hub.docker.com/r/kvanc/tempest_epg

[K-vanc]

idk if this its possible. on tempest configurator , after choose cyclone_site , and you need choose cyclone id, can be complex to navigate and choose the right channel, if you choose a long long cyclone site, that can be easier like a box like the one used on channel configuration

this will be available as of version 1.7

[K-vanc]

Dear Tempest Users;

here is a spoiler alert for next version update

[Toei79]

thats pretty cool feature :)

[K-vanc]

Dear Tempest Users;

I am planning to release new version 1.7.0.0 within next week so this is an important notice for all of your awareness in advance about some critical changes that may effect your setup. I decided to change all CLI command parameters because they are so ugly and unnecessarily long.. To be honest, that commands were from the days that I run Tempest for myself only via curl request(with -F flags) without any setup for cli then I published repo suddenly and couldn't find a chance to modernize them.. Anyway, I think most of you will like new commands so examples for EPG grabbing;

Old one:
php /your/tempest.php engine=Generate tempconfig=test.config.xml createxmlgz=on createinvxmlgz=on

WTF!, isn't it so ugly and long?

New one:
php /your/tempest.php --epg config=test gz invgz

and for channel creation;

Old one:
php /your/tempest.php chengine=Generate siteconfig=test_0 userkey=asdf rm_dupe_id=on rm_dupe_name=on name_modifier=||#addstart#blabla

New one:
php /your/tempest.php --channel config=test_0 ukey=keyvalue dpid dpnm "nmod=||#addstart#blabla"

This new command style will match with existing Tempest update command;

php /your/tempest.php --update

and also will be match with newly introduced xml merger;

php /your/tempest.php --xmlmerge xml=file1 xml=file2 xml=fileN out=merged gz

So I hope you will like new version

[K-vanc]

Yes, Gents;

Finally rv1.7.0.0 has been released. I am waiting for your feedbacks especially for the new XML Merger module

[ghost]

Just tested XML Merger module, working good and fast.
Just for curiosity I tried merging file like
php tempest.php --xmlmerge xml=tempest xml=http://www.test/epg.xml out=merged gz.
Here my second file is web address, can we add downloading feature in it. Like first xml merger will download second file from web or any other sources (not from tempest) and then merger will be worked.

[K-vanc]

hi, xml parameter searches given filename inside /epg/ folder. It is not designed for remote files. Xml merging is actually slow and memory hunger job especially when you have big files. The method I found is fast and low memory solution with local files only. I can merge two >50mb xml files less then half minute with default memory setting. This job takes 13minutes with traditional xml processing methods when you add duplicate xmltv_id into account. The trick here is not reading file so no matter how big it is. Also there is another issue with security. Some php solutions such as url_fopen is disabled by default for %99 of the servers/systems due to security. This cannot be enabled from script side. Normal download methods will read file into memory and allowing such thing will create a security hole for downloading malicious data into your system.
My personal opinion this is not a good idea for many reasons but I will add it to my "MAYBE-TO-DO" list. I think you can do the same from commandline by adding 1 wget/curl line to download file into your /epg/ folder before --xmlmerge line so the risk of installing virus into your system will be your responsibility, not mine 😄

[ghost]

hi, xml parameter searches given filename inside /epg/ folder. It is not designed for remote files. Xml merging is actually slow and memory hunger job especially when you have big files. The method I found is fast and low memory solution with local files only. I can merge two >50mb xml files less then half minute with default memory setting. This job takes 13minutes with traditional xml processing methods when you add duplicate xmltv_id into account. The trick here is not reading file so no matter how big it is. Also there is another issue with security. Some php solutions such as url_fopen is disabled by default for %99 of the servers/systems due to security. This cannot be enabled from script side. Normal download methods will read file into memory and allowing such thing will create a security hole for downloading malicious data into your system. My personal opinion this is not a good idea for many reasons but I will add it to my "MAYBE-TO-DO" list. I think you can do the same from commandline by adding 1 wget/curl line to download file into your /epg/ folder before --xmlmerge line so the risk of installing virus into your system will be your responsibility, not mine 😄

Yeah! It is really fast and consume low memory rather than traditional one.

[Toei79]

no yet installed its been running smooth so far , gonna try thanks

[Toei79]

installed see how goes, old line no work , new one yes replacing on crontab.

[SpitFireRSA]

Dear Tempest Users;

I am planning to release new version 1.7.0.0 within next week so this is an important notice for all of your awareness in advance about some critical changes that may effect your setup. I decided to change all CLI command parameters because they are so ugly and unnecessarily long.. To be honest, that commands were from the days that I run Tempest for myself only via curl request(with -F flags) without any setup for cli then I published repo suddenly and couldn't find a chance to modernize them.. Anyway, I think most of you will like new commands so examples for EPG grabbing;

Old one: php /your/tempest.php engine=Generate tempconfig=test.config.xml createxmlgz=on createinvxmlgz=on

WTF!, isn't it so ugly and long?

New one: php /your/tempest.php --epg config=test gz invgz

and for channel creation;

Old one: php /your/tempest.php chengine=Generate siteconfig=test_0 userkey=asdf rm_dupe_id=on rm_dupe_name=on name_modifier=||#addstart#blabla

New one: php /your/tempest.php --channel config=test_0 ukey=keyvalue dpid dpnm "nmod=||#addstart#blabla"

This new command style will match with existing Tempest update command;

php /your/tempest.php --update

and also will be match with newly introduced xml merger;

php /your/tempest.php --xmlmerge xml=file1 xml=file2 xml=fileN out=merged gz

So I hope you will like new version

Thank you for this reference! Managed to get the EPG updated on a schedule.

Will you be updating the docker to image rv1.7 soon? This is what is currently running: <div id="versx">rv1.6.0.0-2310141014</div>

[K-vanc]

Hi, you can update Tempest without container rebuilt. For cli, use below line;

docker exec -u root yourcontainername php /var/www/html/tempest.php --update

And for Gui, just use below option from your browser;

[SpitFireRSA]

Thanks! I obviously missed this (first coffee of the morning). Thank you for a great tool 👍

[K-vanc]

Dear Tempest Users;

After some passed time without receiving any update, I would like to mention that work is on-progress for the final(planned) evolvement of Tempest as V2.0.0.0 with the last major module add-on.. I will not tell the details but this last module will turn Tempest into an unrivalled grabbing monster(Imagine what is coming since Tempest is already unrivalled as its current form😄). Maybe some of you may know meaning of below diagram;

But, the first and more important part is [ENC][EX] siteconfigs will expire on 03.08.2024 so I will start updating repo for all [ENC][EX] files in the following days since expiry date is less than a month..

Anyway guys, just stay tuned for the coming updates 😄

[Toei79]

looks very intriguing and crazy for sure! plus insane.

[K-vanc]

Expiry dates of all [ENC][EX] siteconfigs are extended for 1 year. Please do not forget to update your local copies before 03.08.2024 which is the last day of old copies'

[K-vanc]

Gents;

It is time to try Tempest's new particle accelerator module "Synchrotron" on rv2.0.0.0. It reduced my daily 315 channels/57 siteconfigs grabbing time from 20 mins to 8-9 mins with its default settings.

[K-vanc]

Hi gents;

This thread is probably reached the end of its life. There are currently too many topics inside and it started to be difficult to find what you are looking so I am closing this issue. New announcements/discussions will be published under "Discussions" section as it should be since the beginning, sorry, this was my first project in Github..

Anyway guys, thanks all of you for your participation here. Your thoughts and suggestions were great and very useful for improving Tempest if you consider several features added after your suggestions here..

[Frost2023]

Hello, dear K-vanc! First of all, let me to thank you for your Tempest EPG! I've got one question for you. Could I have any possibility to create my own channel lineup for Optimum TV USA. If yes, how's then. Thank you for advance anyway!

[K-vanc]

Hi @Frost2023

I just checked that siteconfig and noticed channel creation is not working then checked the website and saw that they are using pdf channel lists now instead of database.

https://static.tvlistings.optimum.net/ool/static/prod/documents/channel-listings/channel-lineup-Brooklyn.pdf

Tempest can parse pdf as well but this is an experimental feature and probably will not encode all properly since there are too many things (bullets, pictures, datas with different fonts/styles, labels etc.) so probably you cannot and I also dont have that much free time to try.. Good part is channel numbers in pdf file, are still valid and can be used for grabbing. So if you want to update, you can download pdf of state you want and update your channel.xml file manually as channel name and channel number.

This is not the best way but there is no chance of automation atm

[Frost2023]

Thank you very much indeed! Flixed.io doesn't work property. Do you have any information will it work? сб, 14 дек. 2024 г., 16:57 Kivanc ***@***.***>:

…

Hi @Frost2023 <https://github.com/Frost2023> I just checked that siteconfig and noticed channel creation is not working then checked the website and saw that they are using pdf channel lists now instead of database. https://static.tvlistings.optimum.net/ool/static/prod/documents/channel-listings/channel-lineup-Brooklyn.pdf Tempest can parse pdf as well but this is an experimental feature and probably will not encode all properly since there are too many things (bullets, pictures, datas with different fonts/styles, labels etc.) so probably you cannot and I also dont have that much free time to try.. Good part is channel numbers in pdf file, are still valid and can be used for grabbing. So if you want to update, you can download pdf of state you want and update your channel.xml file manually as channel name and channel number. This is not the best way but there is no chance of automation atm — Reply to this email directly, view it on GitHub <#17 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A7B4YKZQTXGLKSRYRUOFWKD2FQTMJAVCNFSM6AAAAAAUVRUM2GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNBTGA4TSMRWGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[K-vanc]

I have no idea at all. If you explain what you mean by "not working properly" as which channels, which php/tempest version etc., I can try to check tomorrow. Otherwise, I cannot help with that limited information. Flixed.io server may stop responding and give dummy data if you request too much, maybe it is related as your "properly" word

[Frost2023]

Last date is shown for all channels in all CA and US listings is 26.09.2024. This situation is on Flixed.io website. сб, 14 дек. 2024 г., 17:18 Kivanc ***@***.***>:

…

I have no idea at all. If you explain what you mean by "not working properly" as which channels, which php/tempest version etc., I can try to check tomorrow. Otherwise, I cannot help with that limited information. Flixed.io server may stop responding and give dummy data if you request too much, maybe it is related as your "properly" word — Reply to this email directly, view it on GitHub <#17 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A7B4YK3LLLBNCGJW5BSHJLT2FQV3JAVCNFSM6AAAAAAUVRUM2GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNBTGEYDOMBTG4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[K-vanc]

This is what i mean by dummy response. I will check again tomorrow if something changed but problem(or not problem but way of blocking) comes from their server. Tempest is just getting what they have on their website. I always propose tvguide.com instead of flixed due to this issue.

[Frost2023]

tvguide.com isn't so fast as Flixed.io was. сб, 14 дек. 2024 г., 18:34 Kivanc ***@***.***>:

…

This is what i mean by dummy response. I will check again tomorrow if something changed but problem(or not problem but way of blocking) comes from their server. Tempest is just getting what they have on their website. I always propose tvguide.com instead of flixed due to this issue. — Reply to this email directly, view it on GitHub <#17 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A7B4YK7WJAFKBLHNVYK3J7L2FQ6VZAVCNFSM6AAAAAAUVRUM2GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNBTGEZTKMBUGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[K-vanc]

Use ||#indexonly# command if you dont need details but only care about speed. Check tempestWiki.pdf about how to use

[Frost2023]

Thank you very much! 👍 сб, 14 дек. 2024 г., 18:45 Kivanc ***@***.***>:

…

Use ||#indexonly# command if you dont need details but only care about speed. Check tempestWiki.pdf about how to use — Reply to this email directly, view it on GitHub <#17 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A7B4YKYI2BJOGIQORNOUEKD2FRACLAVCNFSM6AAAAAAUVRUM2GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNBTGEZTSMBVGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>