Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tweaks upgrade - Networks Menu & NTP #807

Merged
merged 14 commits into from
May 15, 2023
Merged

Tweaks upgrade - Networks Menu & NTP #807

merged 14 commits into from
May 15, 2023

Conversation

XK9274
Copy link
Member

@XK9274 XK9274 commented May 2, 2023

This PR adds the following features to Tweaks.

Note: VStudio changed some formatting in source files but it hasn't affected compilation of tweaks.

Network:

  •   - > HTTP Server
    
  •   - > SSH (dropbear) (requires the bin in .tmp_update/bin
    
  •   - > FTP (bftpd)
    
  •   - > Telnet
    
  •   - > WiFi Hotspot
    

Tweaks_000

Emulated time skip moved to new submenu header under System, new System menu structure:

System:

  •   - > Startup...
    
  •   - > Date and time...
    
  •   	   - > Set auto from network
    
  •   	   - > Select timezone
    
  •   	   - > Emulated time skip
    
  •   - > Save and exit when battery <4%
    
  •   - > Vibration intensity
    

Tweaks_001

Adds a smaller hotspot icon which has appeared since using hostapd:

MainUI_000

Global known issues:

  • Main gets a bit sluggish when all services are running (especially opening/closing tweaks) Mostly resolved

HTTP Server

  • Adds the filebrowser menu option already included in onion

Known issues

  • Now binds to 0.0.0.0, no issues with this.

SSH

  • Adds an SSH menu option
  • Needs dropbear in .tmp_update/bin

Known issues

  • none yet encountered.

FTP

  • Adds the filebrowser menu option already included in onion (bftpd)

Known issues

  • Using filezilla as your client needs a slight change or bin files are corrupt on transfer (transfer -> transfer type -> binary)

Telnet

  • Adds telnet option (native telnet client)
  • Runs with telnet -l sh

Known issues

  • It's telnet

Personal hotspot:

  • Adds a personal hotspot option under the Networks top level menu.
  • Modified runtime.sh
  • Modified tweaks source files
  • Added bins / libs
  • Generates logs in .tmp_update/logs/ that are cleared by runtime.sh in clear_logs
  • Creates a hotspot on wlan0
  • Starts/stops based on toggle position (NOT started at boot)
  • If Wifi is off but Hotspot is requested, wifi will be enabled and the hotspot started, services will need to be enabled (FTP/HTTP)
  • Turning on hotspot will turn off NTP but the state will be remembered, if hotspot is disabled in the same session NTP will come back up
  • Retro Netplay tested over hotspot w/ discovery working. (although some roms/cores still have issues)

hostapd config - this is the hotspot config

For basic users:

  • The SSID will be MiyooMiniAP
  • The key will be the serial number without the preceeding 0's and no spaces. (example if the serial number is: 000028887D444810 the key will be 28887D444810) unless they've changed it in the file, in which case it'll be left alone.

For advanced users:

  • SSID/Key can be configured in .tmp_update/config/hostapd.conf
  • Minimum characters for key is 8

dnsmasq config - this is the dhcp server config

For basic users:

  • Gateway: 192.168.100.1 (dummy)
  • MMP Address: 192.168.1.100
  • DHCP range: 192.168.100.101-192.168.100.200 /24

For advanced users:

  • IP and range can be configured in .tmp_update/config/dnsmasq.conf, please follow standard TCP/IP practises

Known issues

I've found by disconnecting/re-connecting a fair few times windows sometimes does not want to connect, forgetting the network then retrying has resolved it.
The hotspot can take a couple of seconds to become "connectable", give it 20-30 seconds if you're having issues then try again

NTP:

NTP

  • Adds an NTP menu option (enable/disable) in System -> Date and time (only shows on MMP)
  • Adds a TZ select (UTC+1/UTC+2 etc)
  • Modified runtime.sh
  • Modified tweaks source files
  • Emulated time skip MUST be turned off!

Known issues

  • There's maybe a better way to do this
  • Doesn't sync across all shells as $TZ doesn't get set by export (can be verified by the /logs/ file or by opening the clock app)
  • Some themes are missing the minus symbol so it will show UTC 0100 instead of UTC-0100
  • NTPD is backwards, UTC-1 becomes UTC+1? worked around
  • File times are sometimes incorrect.

Files changed/added at install/run

  • tweaks
  • runtime.sh

Files changed/added at install/run

new libs to .tmp_update/lib

  • libnl-3.so.200
  • libnl-genl-3.so.200

new bin files to .tmp_update/bin

  • dnsmasq (used to create DHCP server)
  • hostapd (the hotspot daemon)
  • iw (used in runtime.sh to check the mode of wlan0 (managed or ap)

new config files to .tmp_update/config

  • dnsmasq.conf (configures the DHCP server)
  • hostapd.conf (the hotspot daemon, binds wlan0)
  • dhcp.leases (file generated by runtime.sh if not created by dnsmasq, contains lease information)

new logfiles generated in /.tmp_update/logs but cleared by runtime.sh

  • dnsmasq.log (generated by running)
  • network.log

Example log file:

Sun May  7 21:47:35 UTC 2023 FTP: Wifi is turned off, disabling the toggle for FTP and killing the process
Sun May  7 21:47:35 UTC 2023 Telnet: Wifi is turned off, disabling the toggle for Telnet and killing the process
Sun May  7 21:47:47 UTC 2023 Hotspot: requested but WiFi is off, bringing WiFi up now.
Sun May  7 21:47:52 UTC 2023 Hotspot: Started with gateway of: 192.168.100.100, subnet of: 255.255.255.0
Sun May  7 21:48:01 UTC 2023 FTP: Starting bftpd
Sun May  7 21:48:01 UTC 2023 Dropbear: Starting dropbear
Sun May  7 21:48:01 UTC 2023 Telnet: Starting telnet
Sun May  7 21:48:02 UTC 2023 Filebrowser: Starting filebrowser listening on 0.0.0.0 to accept all traffic
Sun May  7 21:48:11 UTC 2023 Hotspot: Something has grabbed wlan0 when we're supposed to be in AP mode, restarting hotspot
Sun May  7 21:48:17 UTC 2023 Hotspot: Started with gateway of: 192.168.100.100, subnet of: 255.255.255.0
Sun May  7 21:48:46 UTC 2023 FTP: Killed
Sun May  7 21:48:46 UTC 2023 Dropbear: Killed
Sun May  7 21:48:46 UTC 2023 Telnet: Killed
Sun May  7 21:48:46 UTC 2023 Filebrowser: Killed
Sun May  7 21:48:46 UTC 2023 Hotspot: Killed
Sun May  7 21:49:02 UTC 2023 FTP: Starting bftpd
Sun May  7 21:49:02 UTC 2023 Dropbear: Starting dropbear
Sun May  7 21:49:02 UTC 2023 Telnet: Starting telnet
Sun May  7 21:49:02 UTC 2023 Filebrowser: Starting filebrowser listening on 0.0.0.0 to accept all traffic
Sun May  7 21:49:07 UTC 2023 Hotspot: Started with gateway of: 192.168.100.100, subnet of: 255.255.255.0
Sun May  7 21:49:57 UTC 2023 Hotspot: Something has grabbed wlan0 when we're supposed to be in AP mode, restarting hotspot
Sun May  7 21:50:02 UTC 2023 Hotspot: Started with gateway of: 192.168.100.100, subnet of: 255.255.255.0
Sun May  7 21:50:04 UTC 2023 Hotspot: Killed
Sun May  7 21:50:04 UTC 2023 NTP: Starting NTP with TZ of UTC-1
Sun May  7 21:50:05 UTC 2023 NTP1: TZ set to UTC-1, Time set to: Sun May  7 21:50:05 UTC 2023 and merged to hwclock, which shows: Sun May  7 21:50:06 2023  0.000000 seconds
Sun May  7 21:50:45 UTC 2023 FTP: Wifi is turned off, disabling the toggle for FTP and killing the process
Sun May  7 21:50:45 UTC 2023 Dropbear: Wifi is turned off, disabling the toggle for dropbear and killing the process
Sun May  7 21:50:45 UTC 2023 Telnet: Wifi is turned off, disabling the toggle for Telnet and killing the process
Sun May  7 21:50:45 UTC 2023 NTP: Wifi is turned off, disabling the toggle for NTP and killing the process
Sun May  7 21:50:45 UTC 2023 Filebrowser: Wifi is turned off, disabling the toggle for HTTP FS and killing the process
Sun May  7 21:51:16 UTC 2023 FTP: Starting bftpd
Sun May  7 21:51:16 UTC 2023 Dropbear: Starting dropbear
Sun May  7 21:51:17 UTC 2023 Telnet: Starting telnet
Sun May  7 21:51:17 UTC 2023 Filebrowser: Starting filebrowser listening on 0.0.0.0 to accept all traffic

XK9274 added 8 commits May 2, 2023 14:04
Tweaks changes:

# This PR adds the following features to Tweaks.

Network:
		- > HTTP Server
		- > SSH (Menu structure built but hidden due to issues with OpenSSH starting the root script on connection and crashing MainUI, this will be added when dropbear comes available)
		- > FTP (bftpd)
		- > Telnet
		- > WiFi Hotspot

Emulated time skip moved to new submenu header under System, new System menu structure:

System:
		- > Startup...
		- > Date and time...
			- > Set auto from network
			- > Select timezone
			- > Emulated time skip
		- > Save and exit when battery <4%
		- > Vibration intensity

# Global known issues:
- Main gets a bit sluggish when all services are running (especially opening/closing tweaks)

## HTTP Server
- Adds the filebrowser menu option already included in onion

# Known issues
- Binds to wlan0 IP address so does not function over hotspot without manual changes

## SSH
- Adds an SSH menu option
- Needs dropbear in .tmp_update/bin

# Known issues
- none yet encountered.

## FTP
- Adds the filebrowser menu option already included in onion (bftpd)

# Known issues
- Using filezilla as your client needs a slight change or bin files are corrupt on transfer (transfer -> transfer type -> binary)

## Telnet
- Adds telnet option (native telnet client)
- Runs with telnet -l sh

# Known issues
- It's telnet

## Personal hotspot:
Personal hotspot
- Adds a personal hotspot option under the Networks top level menu.
- Modified runtime.sh
- Modified tweaks source files
- Added bins / libs
- Generates logs in .tmp_update/logs/ that are cleared by runtime.sh in clear_logs
- Creates a hotspot on wlan1 to be used for any other network features (If the network feature has been built to bind against 0.0.0.0 and NOT the wlan0 address). Currently telnet and bftpd work across the hotspot however filebrowser binds against wlan0 so does not.
- Running on wlan1 gives us the benefit of having both wlan0 and wlan1 running over the same chip. (dual network)
- Starts/stops based on toggle position (persistent across boots)
- No bridging across wlan0 and wlan1

# hostapd config - this is the hotspot config
# For basic users:
- The SSID will be MiyooMiniAP
- The key will be the serial number without the preceeding 0's and no spaces. (example if the serial number is: 000028887D694890 the key will be 28887D694890) unless they've changed it in the file, in which case it'll be left alone.

For advanced users:
- SSID/Key can be configured in .tmp_update/config/hostapd.conf
- Minimum characters for key is 8

# dnsmasq config - this is the dhcp server config
For basic users:
- Gateway IP (The MMP address): 192.168.100.100
- DHCP range: 192.168.100.101-192.168.100.200 /24

For advanced users:
- IP and range can be configured in .tmp_update/config/dnsmasq.conf

# Known issues
I've found by disconnecting/re-connecting a fair few times windows sometimes does not want to connect, forgetting the network then retrying has resolved it.

# Files changed/added at install/run
new libs to .tmp_update/lib
libnl-3.so.200
libnl-genl-3.so.200

new bin files to .tmp_update/bin
dnsmasq
hostapd

new config files to .tmp_update/config
dnsmasq.conf
hostapd.conf
dhcp.leases (file generated by runtime.sh if not created by dnsmasq)

new logfiles generated in /.tmp_update/logs but cleared by runtime.sh
dnsmasq.log (generated by running)

## NTP:

NTP
- Adds an NTP menu option (enable/disable) in System -> Date and time (only shows on MMP)
- Adds a TZ select (UTC+1/UTC+2 etc)
- Modified runtime.sh
- Modified tweaks source files

# Known issues
Doesn't sync across all shells as $TZ doesn't get set by export (can be verified by the /logs/ file or by opening the clock app)
Some themes are missing the minus symbol so it will show UTC  0100 instead of UTC-0100
NTPD is backwards, UTC-1 becomes UTC+1? worked around
File times may still be incorrect, not sure why.

# Files changed/added at install/run
tweaks
runtime.sh

## Example log file:

This log is a cycle of systemboot->disabled all->enabled all->disabled wifi->enabled wifi->enabled all

```FTP: Starting bftpd
Dropbear: Starting dropbear
Filebrowser: Checking for an IP address and trying to start
Hotspot: Key not default, not editing
Hotspot: Started with gateway of: 192.168.100.100, subnet of: 255.255.255.0
Filebrowser: Starting filebrowser with 192.168.1.215
NTP: Starting NTP with TZ of UTC-1
NTP1: TZ set to UTC-1, Time set to: Tue May  2 13:48:25 UTC 2023 and merged to hwclock, which shows: Tue May  2 13:48:26 2023  0.000000 seconds
FTP: Killed
Dropbear: Killed
Telnet: Killed
Filebrowser: Killed
Hotspot: Killed
NTP: Killed by request
FTP: Starting bftpd
Dropbear: Starting dropbear
Telnet: Starting telnet
Filebrowser: Checking for an IP address and trying to start
Filebrowser: Starting filebrowser with 192.168.1.215
Hotspot: Key not default, not editing
Hotspot: Started with gateway of: 192.168.100.100, subnet of: 255.255.255.0
NTP: Starting NTP with TZ of UTC-1
NTP1: TZ set to UTC-1, Time set to: Tue May  2 13:49:15 UTC 2023 and merged to hwclock, which shows: Tue May  2 13:49:16 2023  0.000000 seconds
Hotspot: Wifi is turned off, disabling the toggle for hotspot and killing the process
NTP: Wifi is turned off, disabling the toggle for NTP and killing the process
FTP: Killed
Dropbear: Killed
Telnet: Killed
Filebrowser: Killed
FTP: Wifi is turned off, disabling the toggle for FTP and killing the process
Dropbear: Wifi is turned off, disabling the toggle for dropbear and killing the process
Telnet: Wifi is turned off, disabling the toggle for Telnet and killing the process
Filebrowser: Checking for an IP address and trying to start
Filebrowser: Wifi is turned off, disabling the toggle for HTTP FS and killing the process
Hotspot: Wifi is turned off, disabling the toggle for hotspot and killing the process
FTP: Wifi is turned off, disabling the toggle for FTP and killing the process
Dropbear: Wifi is turned off, disabling the toggle for dropbear and killing the process
Telnet: Wifi is turned off, disabling the toggle for Telnet and killing the process
Filebrowser: Checking for an IP address and trying to start
Filebrowser: Wifi is turned off, disabling the toggle for HTTP FS and killing the process
Hotspot: Wifi is turned off, disabling the toggle for hotspot and killing the process
NTP: Wifi is turned off, disabling the toggle for NTP and killing the process
FTP: Starting bftpd
Dropbear: Starting dropbear
Telnet: Starting telnet
Filebrowser: Checking for an IP address and trying to start
Hotspot: Key not default, not editing
Hotspot: Started with gateway of: 192.168.100.100, subnet of: 255.255.255.0
NTP: Starting NTP with TZ of UTC-1
NTP1: TZ set to UTC-1, Time set to: Tue May  2 13:51:06 UTC 2023 and merged to hwclock, which shows: Tue May  2 13:51:07 2023  0.000000 seconds
Filebrowser: Starting filebrowser with 192.168.1.215```
as per conversation with kebab, gateway being set is causing problems currently so have removed it. The ip address that gets set on wlan1 will now be the DHCP scope -1 (example, 192.168.100.101 first usable address, wlan1 will be set to 192.168.100.100)
- Hotspot now bound to wlan0
- Reproduced steps to get hotspot icon and added to runtime.sh
- Changed filebrowser startup to bind to 0.0.0.0 so we don't need to check/chase IP addresses.
- Tidied comments.
- Commented out check_networking()
- Updated echo's to the logfile with time and date for NTP.
- Moved the starting of the hotspot to a function that's called through check_hotspotstate as we need to chase the startup, cuts down on duped lines
Retroarch will only send a netplay discovery broadcast if there's (any) default via set
Re_add this func. or search crashes on boot, remove the kill from filebrowser
The original hotspot icon is too big for the GUI imo, have rescaled to 35x35 which seems to fit better
Commit description updated w/ new features.

Hotspot is always disabled at boot.

If Wifi is off and hotspot is started, Wifi will be brought up and the label for Wifi will be turned on in main (if you go into wifi though it will kill hotspot until the next state change)

All network services are disabled if wifi is disabled at the first state change (in and out of app) (before it was the second)

Fixed a hang relating to NTP and hotspot

Fixed some processes making Main sluggish relating to wifi

Standard wifi connection reliably comes back up when hotspot is disabled

NTP will now be disabled when hotspot is enabled (pointless having both on taking up resources)
@Aemiii91 Aemiii91 self-requested a review May 7, 2023 21:02
XK9274 added 4 commits May 8, 2023 19:09
- Moved filebrowser branding out of the check_httpserver (Happens once on startup now)
- Moved the check for if Main has taken wlan0 before the hotspot could get it (if switched on)
- Removed the kill for dnsmasq and hostapd in start_hotspot, if they're already running there's not really any need to kill them and restart (if there's hotspot issues they can bounce through tweaks)
- Added check_networking to logs
On initial testing killing dnsmasq and hostapd didn't seem necessary however after leaving running it appears hostapd becomes unstable without this kill (hotspot icon comes on, spotty connection and poor pings)

Reply from 192.168.100.100: bytes=32 time=92ms TTL=64
Reply from 192.168.100.100: bytes=32 time=17ms TTL=64
Request timed out.
Reply from 192.168.100.100: bytes=32 time=38ms TTL=64
Reply from 192.168.100.100: bytes=32 time=29ms TTL=64
Request timed out.
Request timed out.
Request timed out.
Reply from 192.168.100.100: bytes=32 time=19ms TTL=64
Request timed out.
Request timed out.
Request timed out.
Reply from 192.168.100.100: bytes=32 time=20ms TTL=64
Request timed out.
Request timed out.
Reply from 192.168.100.100: bytes=32 time=39ms TTL=64
Reply from 192.168.100.100: bytes=32 time=12ms TTL=64
Reply from 192.168.100.100: bytes=32 time=15ms TTL=64
Reply from 192.168.100.100: bytes=32 time=2ms TTL=64
Request timed out.
Removed a reference that appears in logs falsely specifying the MMP has been set as the gateway
Set password back to default, add OnionOS branding to hotspot name
@Aemiii91 Aemiii91 changed the base branch from dev-V4.2-beta to dev-V4.2-beta-networking May 15, 2023 19:45
@Aemiii91 Aemiii91 merged commit d74b9bc into OnionUI:dev-V4.2-beta-networking May 15, 2023
@Aemiii91 Aemiii91 removed their request for review May 15, 2023 19:51
Aemiii91 added a commit that referenced this pull request May 29, 2023
* fix(nl-lang-short): translate "no thumbnail" to dutch

* Update fuse core (solve joystick issue)

Based on this PR : libretro/fuse-libretro#139
On the previous build, for arcade style games, kempston joystick is pulling the controls to the upper right and pressing fire all the time.
This PR seems to fix it.

* Lower volume at first boot for MMP

* Removing some kill wpa_supplicant in network apps

* OTA Update : now checking github reachable with https protocol instead of icmp

Avoid situations where ICMP protocol is filtered

* Removed playactivity from all launch.sh files

removed "Timer initialisation"
and "Timer registration" from
all launch.sh files in /static/packages/Emu/ and /static/packages/RApp/

* Update Gambatte + a new "GB-Pocket" palette  (04e73b8)

Adding "GB-Pocket" palette (by Jeltron) in "Essentials" section

https://github.com/schmurtzm/gambatte-libretro/tree/advanced-color-organisation    (04e73b8)

* play_activity_ui

* Stay awake while OTA update is running

* Patch `retroarch.cfg` based on previous Onion version

* Fix path

* Fixed formatting for launch.sh files

added newlines to EOF

* Update RA

* Fix deepclean of RA

* Removing too low LCD voltage values.

* Upgrade MainUI binaries to `20230326`

* Update lang

* Migrate usage of `cache2` -> `cache6`

* adding TGB Dual libretro GB core (a6f3018)

Faster Fast Forward for pokemon gamers
Could support GB link ?

* Fix Tweaks LCD ListItem

* Replace SSH/SFTP server (OpenSSH -> dropbear)

- minimal resource usage
- wide compatibility
  * kex: curve25519-sha256,[email protected],ecdh-sha2-nistp521,ecdh-sha2-nistp384,ecdh-sha2-nistp256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1,[email protected]
  * key: ssh-ed25519,ecdsa-sha2-nistp256,rsa-sha2-256,ssh-rsa,ssh-dss
  * cipher: aes128-ctr,aes128-cbc,3des-ctr,3des-cbc
  * mac: hmac-sha1
- single binary for ssh and sftp

SSH: dropbear-2022.83 (https://matt.ucc.asn.au/dropbear/dropbear.html)
  + patch: no auto chdir on connect
  + patch: no new shell session on connect (skip loading /etc/profile)

SFTP: gesftpserver-2 (https://www.greenend.org.uk/rjk/sftpserver/)

source, patches and build configuration available here : https://github.com/anzz1/miyoomini-apps/tree/master/ssh-server

* shorten description

so text does not get cut off

* replace gesftpserver with sftp-server from openssh

its simpler, stabler and uses less memory

* Delete gesftpserver

* Fixed LCD voltage tweak

* Update stop_audioserver.sh

Removing old anti-black screen commands

* Updating gpSP core (f4e65d1)

* Tweaks upgrade

Tweaks changes:

# This PR adds the following features to Tweaks.

Network:
		- > HTTP Server
		- > SSH (Menu structure built but hidden due to issues with OpenSSH starting the root script on connection and crashing MainUI, this will be added when dropbear comes available)
		- > FTP (bftpd)
		- > Telnet
		- > WiFi Hotspot

Emulated time skip moved to new submenu header under System, new System menu structure:

System:
		- > Startup...
		- > Date and time...
			- > Set auto from network
			- > Select timezone
			- > Emulated time skip
		- > Save and exit when battery <4%
		- > Vibration intensity

# Global known issues:
- Main gets a bit sluggish when all services are running (especially opening/closing tweaks)

## HTTP Server
- Adds the filebrowser menu option already included in onion

# Known issues
- Binds to wlan0 IP address so does not function over hotspot without manual changes

## SSH
- Adds an SSH menu option
- Needs dropbear in .tmp_update/bin

# Known issues
- none yet encountered.

## FTP
- Adds the filebrowser menu option already included in onion (bftpd)

# Known issues
- Using filezilla as your client needs a slight change or bin files are corrupt on transfer (transfer -> transfer type -> binary)

## Telnet
- Adds telnet option (native telnet client)
- Runs with telnet -l sh

# Known issues
- It's telnet

## Personal hotspot:
Personal hotspot
- Adds a personal hotspot option under the Networks top level menu.
- Modified runtime.sh
- Modified tweaks source files
- Added bins / libs
- Generates logs in .tmp_update/logs/ that are cleared by runtime.sh in clear_logs
- Creates a hotspot on wlan1 to be used for any other network features (If the network feature has been built to bind against 0.0.0.0 and NOT the wlan0 address). Currently telnet and bftpd work across the hotspot however filebrowser binds against wlan0 so does not.
- Running on wlan1 gives us the benefit of having both wlan0 and wlan1 running over the same chip. (dual network)
- Starts/stops based on toggle position (persistent across boots)
- No bridging across wlan0 and wlan1

# hostapd config - this is the hotspot config
# For basic users:
- The SSID will be MiyooMiniAP
- The key will be the serial number without the preceeding 0's and no spaces. (example if the serial number is: 000028887D694890 the key will be 28887D694890) unless they've changed it in the file, in which case it'll be left alone.

For advanced users:
- SSID/Key can be configured in .tmp_update/config/hostapd.conf
- Minimum characters for key is 8

# dnsmasq config - this is the dhcp server config
For basic users:
- Gateway IP (The MMP address): 192.168.100.100
- DHCP range: 192.168.100.101-192.168.100.200 /24

For advanced users:
- IP and range can be configured in .tmp_update/config/dnsmasq.conf

# Known issues
I've found by disconnecting/re-connecting a fair few times windows sometimes does not want to connect, forgetting the network then retrying has resolved it.

# Files changed/added at install/run
new libs to .tmp_update/lib
libnl-3.so.200
libnl-genl-3.so.200

new bin files to .tmp_update/bin
dnsmasq
hostapd

new config files to .tmp_update/config
dnsmasq.conf
hostapd.conf
dhcp.leases (file generated by runtime.sh if not created by dnsmasq)

new logfiles generated in /.tmp_update/logs but cleared by runtime.sh
dnsmasq.log (generated by running)

## NTP:

NTP
- Adds an NTP menu option (enable/disable) in System -> Date and time (only shows on MMP)
- Adds a TZ select (UTC+1/UTC+2 etc)
- Modified runtime.sh
- Modified tweaks source files

# Known issues
Doesn't sync across all shells as $TZ doesn't get set by export (can be verified by the /logs/ file or by opening the clock app)
Some themes are missing the minus symbol so it will show UTC  0100 instead of UTC-0100
NTPD is backwards, UTC-1 becomes UTC+1? worked around
File times may still be incorrect, not sure why.

# Files changed/added at install/run
tweaks
runtime.sh

## Example log file:

This log is a cycle of systemboot->disabled all->enabled all->disabled wifi->enabled wifi->enabled all

```FTP: Starting bftpd
Dropbear: Starting dropbear
Filebrowser: Checking for an IP address and trying to start
Hotspot: Key not default, not editing
Hotspot: Started with gateway of: 192.168.100.100, subnet of: 255.255.255.0
Filebrowser: Starting filebrowser with 192.168.1.215
NTP: Starting NTP with TZ of UTC-1
NTP1: TZ set to UTC-1, Time set to: Tue May  2 13:48:25 UTC 2023 and merged to hwclock, which shows: Tue May  2 13:48:26 2023  0.000000 seconds
FTP: Killed
Dropbear: Killed
Telnet: Killed
Filebrowser: Killed
Hotspot: Killed
NTP: Killed by request
FTP: Starting bftpd
Dropbear: Starting dropbear
Telnet: Starting telnet
Filebrowser: Checking for an IP address and trying to start
Filebrowser: Starting filebrowser with 192.168.1.215
Hotspot: Key not default, not editing
Hotspot: Started with gateway of: 192.168.100.100, subnet of: 255.255.255.0
NTP: Starting NTP with TZ of UTC-1
NTP1: TZ set to UTC-1, Time set to: Tue May  2 13:49:15 UTC 2023 and merged to hwclock, which shows: Tue May  2 13:49:16 2023  0.000000 seconds
Hotspot: Wifi is turned off, disabling the toggle for hotspot and killing the process
NTP: Wifi is turned off, disabling the toggle for NTP and killing the process
FTP: Killed
Dropbear: Killed
Telnet: Killed
Filebrowser: Killed
FTP: Wifi is turned off, disabling the toggle for FTP and killing the process
Dropbear: Wifi is turned off, disabling the toggle for dropbear and killing the process
Telnet: Wifi is turned off, disabling the toggle for Telnet and killing the process
Filebrowser: Checking for an IP address and trying to start
Filebrowser: Wifi is turned off, disabling the toggle for HTTP FS and killing the process
Hotspot: Wifi is turned off, disabling the toggle for hotspot and killing the process
FTP: Wifi is turned off, disabling the toggle for FTP and killing the process
Dropbear: Wifi is turned off, disabling the toggle for dropbear and killing the process
Telnet: Wifi is turned off, disabling the toggle for Telnet and killing the process
Filebrowser: Checking for an IP address and trying to start
Filebrowser: Wifi is turned off, disabling the toggle for HTTP FS and killing the process
Hotspot: Wifi is turned off, disabling the toggle for hotspot and killing the process
NTP: Wifi is turned off, disabling the toggle for NTP and killing the process
FTP: Starting bftpd
Dropbear: Starting dropbear
Telnet: Starting telnet
Filebrowser: Checking for an IP address and trying to start
Hotspot: Key not default, not editing
Hotspot: Started with gateway of: 192.168.100.100, subnet of: 255.255.255.0
NTP: Starting NTP with TZ of UTC-1
NTP1: TZ set to UTC-1, Time set to: Tue May  2 13:51:06 UTC 2023 and merged to hwclock, which shows: Tue May  2 13:51:07 2023  0.000000 seconds
Filebrowser: Starting filebrowser with 192.168.1.215```

* Fix Netplay

This is a workaround to avoid a Retroarch issue :
libretro/RetroArch#15248

When the RA issue will be fixed we will able to restore savestate thumbnails (which are still useful for browsing savestates in RA).

* Remove gateway option

as per conversation with kebab, gateway being set is causing problems currently so have removed it. The ip address that gets set on wlan1 will now be the DHCP scope -1 (example, 192.168.100.101 first usable address, wlan1 will be set to 192.168.100.100)

* Various changes, fixes and optimisations

- Hotspot now bound to wlan0
- Reproduced steps to get hotspot icon and added to runtime.sh
- Changed filebrowser startup to bind to 0.0.0.0 so we don't need to check/chase IP addresses.
- Tidied comments.
- Commented out check_networking()
- Updated echo's to the logfile with time and date for NTP.
- Moved the starting of the hotspot to a function that's called through check_hotspotstate as we need to chase the startup, cuts down on duped lines

* Adds a dummy gateway, fixes retroarch netplay discovery

Retroarch will only send a netplay discovery broadcast if there's (any) default via set

* Remove "break"

* Re-Add check_networking

Re_add this func. or search crashes on boot, remove the kill from filebrowser

* Rescaled hotspot icon

The original hotspot icon is too big for the GUI imo, have rescaled to 35x35 which seems to fit better

* Adds new features and fixes

Commit description updated w/ new features.

Hotspot is always disabled at boot.

If Wifi is off and hotspot is started, Wifi will be brought up and the label for Wifi will be turned on in main (if you go into wifi though it will kill hotspot until the next state change)

All network services are disabled if wifi is disabled at the first state change (in and out of app) (before it was the second)

Fixed a hang relating to NTP and hotspot

Fixed some processes making Main sluggish relating to wifi

Standard wifi connection reliably comes back up when hotspot is disabled

NTP will now be disabled when hotspot is enabled (pointless having both on taking up resources)

* Fix state.json main menu ID + go to main menu instead of app page

* Switch to using file flag for mute instead of system.json

* Fixes & Optimisations

- Moved filebrowser branding out of the check_httpserver (Happens once on startup now)
- Moved the check for if Main has taken wlan0 before the hotspot could get it (if switched on)
- Removed the kill for dnsmasq and hostapd in start_hotspot, if they're already running there's not really any need to kill them and restart (if there's hotspot issues they can bounce through tweaks)
- Added check_networking to logs

* Leftover return

* Re-adding hostapd and dnsmasq kills

On initial testing killing dnsmasq and hostapd didn't seem necessary however after leaving running it appears hostapd becomes unstable without this kill (hotspot icon comes on, spotty connection and poor pings)

Reply from 192.168.100.100: bytes=32 time=92ms TTL=64
Reply from 192.168.100.100: bytes=32 time=17ms TTL=64
Request timed out.
Reply from 192.168.100.100: bytes=32 time=38ms TTL=64
Reply from 192.168.100.100: bytes=32 time=29ms TTL=64
Request timed out.
Request timed out.
Request timed out.
Reply from 192.168.100.100: bytes=32 time=19ms TTL=64
Request timed out.
Request timed out.
Request timed out.
Reply from 192.168.100.100: bytes=32 time=20ms TTL=64
Request timed out.
Request timed out.
Reply from 192.168.100.100: bytes=32 time=39ms TTL=64
Reply from 192.168.100.100: bytes=32 time=12ms TTL=64
Reply from 192.168.100.100: bytes=32 time=15ms TTL=64
Reply from 192.168.100.100: bytes=32 time=2ms TTL=64
Request timed out.

* Draw a low battery icon instead of red rectangle

TO DO :
- Do not display it when we are in MainUI (we already have a battery level displayed theorically)
- Send the real battery level in parameter

* New MainUI cuts

* Removed reference to gateway in runtime.sh

Removed a reference that appears in logs falsely specifying the MMP has been set as the gateway

* Adding Scraper for Onion

Additions & modifications :
------------------------------
- Scraper.sh a launcher script in GLO to run menu.sh, also in charge to run the background scraping.
- menu.sh : a frontend for the scraper options.
- scrap_retroarch.sh, scrap_screenscraper.sh and scrap_launchbox.sh are 3 scripts which can be launched independently from menu.sh. scrap_screenscraper.sh "systemfoldername" ["RomPath"] For example : "scrap_screenscraper.sh GB". (Credits Schmurtz with a great help from EmuDeck Team !)
- scraper.json : a configuration file for scraping settings and screenscraper account.
- removing the tee in runtime.sh, the command is not compatible with process in brackground which is required for background scraping.
- shellect script modification to have longer lines.

New programs added :
-------------------------
- pressMenu2Kill (credits Schmurtz) : allows to kill the application passed in parameter by pressing the menu button. Used to kill st / scraping menu when pressing menu button.
- jpg2png (credits eggs) : allows to convert a jpg to png and to resize it at the same time.
- pngScale (credits eggs) : allows to resize a png file.

video demo :
--------------
 https://youtu.be/lOMP0ozb0I0

* Update menu.sh

Easier screenscraper account configuration.

* 🎨 apply clang-format changes

* Fix main menu ID

* Fix hatari save states

* Redirect no longer necessary

* Add default MainUI keyboard skin

* Update hostapd.conf

Set password back to default, add OnionOS branding to hotspot name

* typo on menu.sh from Scraper

* Create pl.lang.short

* Create pl.lang

* Tweaks upgrade - Networks Menu & NTP (#807)

* 🎨 apply clang-format changes

* Cleanup `runtime.sh`

* 🎨 apply clang-format changes

* chmod new script

* Fix script

* Add network icon

* Test network settings changed

* Include "date time" menu in network changes

* Restored hotspot functionality. Restored SSH functionality.

update_networking.sh changes:
Added `check_hotspotstate` to main loop.
Added a check in `check_wifi` to test hotspot state.

Info: When WiFi is started at the same time as the hotspot, wpa_supplicant is quicker to grab wlan0 which results in hotspot failing to come up

Added dropbear to .tmp_update/bin from this PR:#790 to restore SSH functionality and break the SSHState loop pending review of PR.

* Update nl.lang

* Added translation for string 155

* Scraper : filter .miyoocmd and .cfg files

* Pushing a fix for hotspot stability

- update_networking now sourced in runtime.sh
- check_hotspotstate now called in runtime loop
- check_hotspotstate and start_hotspot merged with old/new config to provide recovery after main relaunch.
- start wifi on hotspot launch removed, needs reworking for new layout
- serialnumber WPA code commented out following conversation with schmurtz, see chat. password now `onionos!` (warn: has to be 8 chars!)
- some NTP code moved to a func. as duplicated
- please note hotspot is still killed sometimes when mainui relaunches (during a 'longer' loading screen).

Steps to test full cycle:

- Turn on hotspot
- Observe hotspot icon appears, hotspot shows in other devices & is connectable (10-20 seconds wait)
- Open search with x (or app) - close app
- Observe MainUI relaunch
- Observe hotspot drops but recovers
- Turn hotspot off, observe wifi recovers
- Turn hotspot on
- Turn hotspot off
- Observe all hostapd/dnsmasq processes dead.

* Remove sleep

Removed sleep from NTP - not needed anymore.
Removed backgrounded task.

* Update pl.lang

* Hotspot rework

Commit will change how hotspot functions.

Hotspot has been moved (back) to wlan1 which provides the below fixes but comes with its own issues. MainUI doesn't like the hotspot, we're trying to work around it the best we can.

Important notes:
- Hotspot no longer drops coming out of an app. This was caused by MainUI not detecting wpa_supplicant and restarting wifi. Leaving the supp running
- The hotspot icon has been taken away by MainUI for now..
- Allows for further implementation features such as dual network support
- Gateway set to MMP once again with an ip route, this provides better discovery but has the side-effect of interface bridging.
- Netplay has been tested on hotspot, working fine PC -> MMP but MMP -> MMP the following process must be followed (in my testing, needs validating by someone with 2 mmps)
- Hotspot can still take up to 30 seconds to become connectable.

MMP1: Start your hotspot
MMP2: Connect to the hotspot
MMP1: Start a netplay session on your chosen game
MMP2: Start retroarch from the apps menu item
MMP2: Load a game, any game (but same core)
MMP2: Go back into the menu and select netplay, search LAN, then join.

MMP -> PC is mostly unaffected by this issue. I've had MMP hosted games easily joined by PC RA and vice versa.

All other features (Telnet, SSH, FTP, Filebrowser) are all confirmed working on standard WiFi and on hotspot.

* Fix Netplay for PicoDrive (2nd gamepad)

* Enable tgbdual GB "link cable emulation" by default.

Useful to use Netplay. Good cross platform compatibility.

* Removing functions that were moved to update_networking.sh

* update EasyRPG core to 0.8.0.0

From :
https://community.easyrpg.org/t/arm-binaries-for-easyrpg-player/1250

* Bftpd and Filebrowser bins

Moved Filebrowser and bftpd bins into .tmp_update, made associated changes in update_networking.sh and runtime.sh & tested. bftpd.conf file altered to set logging location etc. Tested across 3 mmps. (Noted in testing sometimes NTP fails on the first attempt, verify with network.log in .tmp_update/logs)

* Update overlays

Co-authored-by: @Jeltr0n

---------

Co-authored-by: nil <[email protected]>
Co-authored-by: xZliman <>
Co-authored-by: Schmurtz <[email protected]>
Co-authored-by: Awkwabear <[email protected]>
Co-authored-by: anzz1 <[email protected]>
Co-authored-by: XK <[email protected]>
Co-authored-by: Sichroteph <[email protected]>
Co-authored-by: Aemiii91 <[email protected]>
Co-authored-by: Saad Asif <[email protected]>
Co-authored-by: Assassins_78 <[email protected]>
Co-authored-by: ronvandegraaf <[email protected]>
@Aemiii91 Aemiii91 added the enhancement New or improved feature label Oct 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New or improved feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants