| ff_ | with 35000 iops it could works like RAM, so, n900 with extremely expensive 32GB RAM/ROM :) | 00:05 |
|---|---|---|
| L29Ah | 23:15:04]<KotCzarny> most noname cards have iops in 0.1-1/s range | 00:05 |
| L29Ah | what | 00:05 |
| ff_ | I couldn't be surprised if Chinese people could produce something like that | 00:29 |
| ff_ | anyone know speed of microSD data bus in n900? | 00:33 |
| ff_ | maybe it's not worth the bother with fast microSD | 00:34 |
| Maxdamantus | I'm getting around 12 MB/s read. | 00:36 |
| Maxdamantus | I have a feeling that the maximum speed does kind of matter. | 00:36 |
| ff_ | what are specs of sd? | 00:37 |
| Maxdamantus | iirc I would only get around 4 MB/s on a lower-end Kingston microSD, which was rated at something like 30 MB/s. | 00:37 |
| ff_ | speed of read or write? | 00:38 |
| Maxdamantus | SanDisk 128GB Ultra® microSDXC™ UHS-I Class 10 card | 00:38 |
| Maxdamantus | SD cards don't really have a particular speed. | 00:39 |
| Maxdamantus | It depends on things like how the controller has laid out the data. | 00:40 |
| ff_ | with it you get 12MB/s in n900? | 00:41 |
| Maxdamantus | Yes. | 00:41 |
| Maxdamantus | 236978176 bytes (237 MB) copied, 20.236 s, 11.7 MB/s | 00:41 |
| ff_ | and how with write? | 00:41 |
| Maxdamantus | write is trickier to test. | 00:42 |
| Maxdamantus | can try after remounting with -o sync, I guess. | 00:43 |
| ff_ | not pressure, I'm just curious | 00:44 |
| Maxdamantus | I'm getting 10.6 MB/s writing to ext4 mounted with -o sync using dd with conv=fsync on | 00:45 |
| Maxdamantus | (so that's to a filesystem, not just a block device) | 00:45 |
| Maxdamantus | though I'm only writing zeroes, dunno if the card does something special with that. | 00:45 |
| Maxdamantus | Well, I imagine the card should handle more than 10.6 MB/s anyway .. it'll at least be pushing that 10.6 MB/s in SDIO packets. | 00:46 |
| ff_ | I don't know also, thanks for real data with n900 | 00:47 |
| Maxdamantus | The sync options are also going to usually reduce overall throughput | 00:47 |
| ff_ | it seems that n900 has 10MB/s speed limit on sd bus | 00:48 |
| Maxdamantus | These speeds are all sequential btw, not random. | 00:48 |
| ff_ | random should be lower than this 10MB/s cameramemoryspeed.com/reviews/sd-cards/sandisk-extreme-plus-95-90-mbs-64gb-micro-sdxc-memory-card/ | 00:50 |
| ff_ | 9MB/s for read and 3MB/s for write if that benchmark is with the same class card | 00:51 |
| ff_ | microSDXC UHS-I U3, I'm not sure for that U3 symbol | 00:55 |
| *** Oksanaa has quit IRC | 00:55 | |
| *** shentey has quit IRC | 00:56 | |
| *** Oksanaa has joined #maemo | 00:57 | |
| *** ff_ has quit IRC | 01:14 | |
| *** jskarvad has quit IRC | 01:15 | |
| DocScrutinizer05 | a pty ff_ left. Was absolutely right it doesn't really make sense to bother, since databus is limited to 4bit buswidth * (iirc) 40MHz clockrate max, so 20MB/s | 01:34 |
| DocScrutinizer05 | *might* even be just 25MHz | 01:35 |
| Oksanaa | Question is, which cards actually work at maximum limit of the databus. | 01:35 |
| DocScrutinizer05 | all | 01:38 |
| DocScrutinizer05 | :-) | 01:38 |
| *** RzR has quit IRC | 01:38 | |
| Maxdamantus | except for my Kingston one. | 01:38 |
| DocScrutinizer05 | it's a pretty ancient databus protocol. Recent Ultra card only work since they support legacy protocols, the OMAP3 doesn't support those new speeds | 01:39 |
| DocScrutinizer05 | UHS-I and UHS-II are not supported by OMAP3 | 01:39 |
| DocScrutinizer05 | for write, the real show stopper is page erase needed to rewrite a block with new data. It can take up to some 100s of MILLIseconds | 01:40 |
| DocScrutinizer05 | at very least 10ms | 01:41 |
| DocScrutinizer05 | per page erase | 01:41 |
| Oksanaa | Is it possible to have faster databus with OMAP3? And what is page erase, is it like going over the card with eraser to be sure that write to card goes well? | 01:42 |
| DocScrutinizer05 | yes @ #2 | 01:43 |
| DocScrutinizer05 | no @ #1, it's a physical interface that can't do any faster | 01:43 |
| DocScrutinizer05 | OMAP has a 4bit and a 8bit storage bus, both with same max clock frequency. In N900 the 8bit is for eMMC | 01:44 |
| Oksanaa | Like, would Neo900 be able to have faster databus than N900? | 01:44 |
| *** ketas is now known as ketas- | 01:44 | |
| DocScrutinizer05 | no | 01:44 |
| DocScrutinizer05 | alas not | 01:44 |
| DocScrutinizer05 | that's also one killer argument why Neo900 must have eMC despite many people would prefer two uSD slots: eMMC supports 8bit | 01:45 |
| Oksanaa | About page erase: is it possible to do it beforehand, like use idle time to erase empty blocks on the card so that by the time a write is needed, card already has several blocks ready? | 01:46 |
| Maxdamantus | I imagine that's something the card will already do. | 01:46 |
| Oksanaa | uSD slot supports only 4bit? | 01:46 |
| Maxdamantus | Hosts don't interact with SD cards as flash devices. | 01:46 |
| Maxdamantus | That's all stuff that's handled by the computer inside the SD card. | 01:46 |
| Oksanaa | Maxdamantus: that would be neat, if it's implemented already. | 01:47 |
| *** ketas has joined #maemo | 01:48 | |
| DocScrutinizer05 | (do it beforehand) that's how all slightly smarter cards work, as long as the OS supports TRUNCATE | 01:49 |
| Maxdamantus | TRUNCATE? Do you mean TRIM? | 01:49 |
| Maxdamantus | if so, that has nothing no do with SD cards. | 01:50 |
| DocScrutinizer05 | the card needs to know which blocks can get erased beforehand, even writing all zeroes to a whole page doesn't automatically tell card "this is no data, this is empty storage free to use" | 01:50 |
| Maxdamantus | well, that's not what SD cards typically do. | 01:50 |
| DocScrutinizer05 | uSd has only 4 bit data, yes. count the pins on your uSD card ;-) | 01:51 |
| DocScrutinizer05 | you need GND, VCC, clock r/w whatnot else | 01:51 |
| Maxdamantus | DocScrutinizer05: can you provide a link to some information about this magical "TRUNCATE" thing you're talking about? | 01:51 |
| Maxdamantus | or is this another case where you're just saying things about technologies you don't understandL | 01:51 |
| DocScrutinizer05 | Oksanaa: https://wiki.debian.org/SSDOptimization | 01:55 |
| Maxdamantus | We're talking about SD cards, not SSDs. | 01:55 |
| Maxdamantus | SD cards don't have TRIM (or "TRUNCATE" if you like making up names for things) | 01:55 |
| DocScrutinizer05 | basically all SSD rationale also applies to SD cards | 01:56 |
| Maxdamantus | SD is a completely different protocol. | 01:56 |
| Maxdamantus | and it doesn't have anything like TRIM in it. | 01:56 |
| DocScrutinizer05 | sorry, DISCARD, not TRUNCATE | 01:56 |
| Maxdamantus | apparently it's actually ERASE | 01:58 |
| DocScrutinizer05 | the best way is to actually never use a maybe 10% of the SD card, by downsizing the partition(s) | 02:02 |
| DocScrutinizer05 | once the system wrote to all available blocks (and didn't issue a DISCARD or whatever the equivalent name on SD bus, for all the no longer needed blocks, and made sure the non-used blocks concatenate to a whole erase page), the card will significantly drop on write performance | 02:04 |
| *** sunshavi has joined #maemo | 02:04 | |
| Oksanaa | So, use up to 90% or so of the card. | 02:05 |
| DocScrutinizer05 | since the card controller can't fortune tell which pages to erase "preemptively" | 02:05 |
| DocScrutinizer05 | card always have a few hidden spare pages, but not that many | 02:06 |
| DocScrutinizer05 | Oksanaa: (90%) I mere guess by me, but sounds good | 02:06 |
| Oksanaa | So, if somebody made a card with high enough quantity of hidden spare pages, that would be easier on user. | 02:07 |
| DocScrutinizer05 | yes | 02:08 |
| Maxdamantus | Well, presumably it'll usually have around 73 MB per GB of spare blocks. | 02:08 |
| Maxdamantus | if its capacity is given in GB. | 02:09 |
| Maxdamantus | since 1 GiB is a bit more than 73 MB more than a GB. | 02:09 |
| Maxdamantus | flash usually comes in a power-of-2 number of bits, then the SD card will offer a power-of-2 number of GBs or MBs. | 02:10 |
| DocScrutinizer05 | maybe interesting (didn't read it completely): https://blogofterje.wordpress.com/2012/01/14/optimizing-fs-on-sd-card/ | 02:11 |
| *** sunshavi has quit IRC | 02:14 | |
| DocScrutinizer05 | https://haydenjames.io/increase-performance-lifespan-ssds-sd-cards/ here it's called TRIM | 02:16 |
| Juesto | Do compactation? | 02:16 |
| DocScrutinizer05 | though >>Therefore, when you delete a file, your SSD is now able to write data to blocks as if it they were brand new without having to perform the cumbersome deletion process.<< obviously is utter bullshit | 02:16 |
| *** DarioAlejandro has quit IRC | 02:25 | |
| DocScrutinizer05 | MEH! I need to avoid looking into chanlog, always an annoyance to read that crap my /ignore luckily usually saves me to read | 02:27 |
| *** sunshavi has joined #maemo | 02:30 | |
| DocScrutinizer05 | Juesto: one of the major problems about uSD embedded controller is: it doesn't know about ext-fs or anything else than VFAT basically. On ext[234] you have inodes scattered all over the complete storage | 02:31 |
| DocScrutinizer05 | it's assumed that the SD controller even knows about File allocation Table in VFAT and optimizes using VFAT properties it knows about. Can't do same for any other filesystem | 02:32 |
| *** dafox has joined #maemo | 02:33 | |
| DocScrutinizer05 | Juesto: anyway SSD/SD analysis and optimization is absolutely non-trivial, due to the complex way a flash storage works and the obscure SEKRIT sauce the uSD controller adds on top | 02:36 |
| luke-jr | Maxdamantus: SD cards have a virtual block layer on top of the real flash, which adds overhead for remapping bad blocks | 02:36 |
| DocScrutinizer05 | that's why for really tight control and fast r/w access you want NAND and a filesystem like ubifs | 02:36 |
| DocScrutinizer05 | oh hi luke-jr :-) | 02:37 |
| luke-jr | hi DocScrutinizer05 | 02:37 |
| * DocScrutinizer05 seems to recall there were plans or even initiatives to reflash the uSd controller firmware to allow using the uSD like it was normal un-managed NAND storage | 02:38 | |
| *** ketas- has quit IRC | 02:39 | |
| DocScrutinizer05 | obviously such a project would be highly brand&make specific on the particular uSD | 02:39 |
| DocScrutinizer05 | Oksanaa: re "90%" don't get fooled: see https://www.sandisk.com/home/memory-cards/sd-cards/extremeplus-sd-uhs-i "DISCLOSURES: ... 1MB=1,000,000 bytes. ** 1GB=1,000,000,000 bytes" | 02:50 |
| DocScrutinizer05 | *plus*: "Actual user storage less." | 02:51 |
| DocScrutinizer05 | so a 32GB uSD has a 32E9 bytes, minus a non-marginal fraction for overhead like spare blocks and prolly also flash controller metadata etc | 02:55 |
| *** Kabouik_ has joined #maemo | 02:58 | |
| *** DarioAlejandro has joined #maemo | 03:02 | |
| *** florian has quit IRC | 03:07 | |
| *** dafox has quit IRC | 03:12 | |
| *** cyphase has quit IRC | 03:16 | |
| DocScrutinizer05 | AAAH and there's the term I missed: FTL aka Flash Translation Layer (aka "the uSD embedded controller) http://www.linux-mtd.infradead.org/doc/ubifs.html | 03:19 |
| DocScrutinizer05 | s/"the uSD embedded controller("the uSD embedded flash controller"/ | 03:19 |
| DocScrutinizer05 | http://www.linux-mtd.infradead.org/doc/ubifs.html#L_raw_vs_ftl | 03:21 |
| Maxdamantus | 13:55:39 < DocScrutinizer05> so a 32GB uSD has a 32E9 bytes, minus a non-marginal fraction for overhead like spare blocks and prolly also flash controller metadata etc | 03:21 |
| Maxdamantus | No. | 03:21 |
| *** cyphase has joined #maemo | 03:21 | |
| Maxdamantus | It has 32 GiB inside, but provides at least 30 GB that you can write to. | 03:21 |
| Maxdamantus | You practically can't get 32 GB in flash chips .. a 32 GB SD card will have 32 GiB of flash inside, but the block device you end up with will be around 32 GB. | 03:23 |
| Maxdamantus | Hm, actually, my 128 GB card only has 127865454592 bytes. | 03:25 |
| DocScrutinizer05 | the trick with optimizing uSD is:http://www.linux-mtd.infradead.org/faq/general.html#L_mtd_vs_hdd you have an interface/protocol for a block device (left side of table, with extensions like TRIM for example), but you try to implement a "usage pattern" aka filesystem and driver that takes care about properties described on right side half of the table | 03:26 |
| *** Kabouik_ has quit IRC | 03:27 | |
| DocScrutinizer05 | you need a priori knowledge (or assumptions) about how the FTL firmware works and what it does in response to what you do on the block device interface | 03:27 |
| Maxdamantus | The eMMC on my N900 has 32015122432 bytes though. | 03:33 |
| DocScrutinizer05 | http://www.linux-mtd.infradead.org/faq/general.html#L_ext2_mtd >>But in many cases using mtdblock is a very bad idea because what it basically does if you change any sector of your mtdblockX device, it reads the whole corresponding eraseblock into the memory, erases the eraseblock, changes the sector in RAM, and writes the whole eraseblock back. This is very straightforward. If you have a power failure when the eraseblock is being erased, | 03:33 |
| DocScrutinizer05 | you lose all the block device sectors in it. The flash will likely decay soon because you will wear few eraseblocks out - most probably those ones which contain FAT/bitmap/inode table/etc.<< | 03:33 |
| DocScrutinizer05 | eraseblock (I called it erase page, sorry) is as large as several dozen or even hundreds of kB | 03:34 |
| DocScrutinizer05 | and to change a singke bit in it, you need to read modify erase write the whole eraseblock | 03:35 |
| Maxdamantus | DocScrutinizer05: yeah, that's why you shouldn't use something like mtdblock, as mentioned above. | 03:35 |
| Maxdamantus | DocScrutinizer05: something sensible would keep writing to new blocks. | 03:35 |
| Maxdamantus | that's basically the principle behind log-based filesystems. | 03:36 |
| Maxdamantus | The FTLs in SD cards probably do something similar. | 03:36 |
| DocScrutinizer05 | *unless* the bit change is 0->1, assuming all-0s is the result of block-erase (which takes several milliseconds as I already explained) | 03:36 |
| Maxdamantus | https://en.wikipedia.org/wiki/List_of_log-structured_file_systems | 03:37 |
| Maxdamantus | The filesystems at the bottom are all the standard ones for flash. | 03:38 |
| Maxdamantus | including UBIFS, which is used by maemo. | 03:38 |
| DocScrutinizer05 | so stuff like FTL and ubifs implements very nifty ways to not always read-modify-write for a few bytes that need to be changed, rather it uses journals etc that simply get extended at end, writing to the already/still all-0 region of a erase block | 03:38 |
| Maxdamantus | an SD card or MMC has to do the same sort of stuff, but it just provides a block device instead of a filesystem. | 03:38 |
| DocScrutinizer05 | and TRIM deals with telling FTL that a complete eraseblock is 'clean' again and ready to get erased when there's some spare time in background for doing so | 03:39 |
| DocScrutinizer05 | (simplified image, since TRIM doesn't really care about erase blocks, it has no notion about them afaik. It's up to the FTL to deduce erase blocks needing a background erase, from what fs blocks (512) TRIM told the FTL are unused | 03:43 |
| DocScrutinizer05 | ) | 03:43 |
| *** Pali has quit IRC | 04:01 | |
| *** Oksanaa has left #maemo | 04:11 | |
| *** RedW has quit IRC | 05:43 | |
| *** RedW has joined #maemo | 05:43 | |
| *** RedW has quit IRC | 05:50 | |
| *** utanapischti has joined #maemo | 05:51 | |
| *** drawkula has quit IRC | 05:51 | |
| *** F3l1x_10m is now known as Yuri_10m | 07:05 | |
| *** DocScrutinizer05 has quit IRC | 07:29 | |
| *** DocScrutinizer05 has joined #maemo | 07:29 | |
| *** chainsawbike has quit IRC | 07:47 | |
| *** chainsawbike has joined #maemo | 07:49 | |
| KotCzarny | l29ah: yes, see above eraseblock explanation | 08:30 |
| KotCzarny | sd cards were created for mobile capture devices | 08:30 |
| KotCzarny | nowadays people started using them as rootfs etc, so sandisk/samsung started optimizing 4k random r/w | 08:31 |
| KotCzarny | if you want to measure correctly run iozone test, you will see for yourlsef | 08:32 |
| *** RedW has joined #maemo | 09:28 | |
| *** florian has joined #maemo | 10:15 | |
| *** Kabouik_ has joined #maemo | 13:08 | |
| *** Pali has joined #maemo | 13:15 | |
| *** xorly has joined #maemo | 13:16 | |
| *** rosseaux has joined #maemo | 13:28 | |
| *** florian has quit IRC | 13:41 | |
| *** Kabouik_ has quit IRC | 14:45 | |
| *** Kabouik_ has joined #maemo | 15:34 | |
| *** ced117_ is now known as ced117 | 15:54 | |
| *** ced117 has joined #maemo | 15:54 | |
| *** florian has joined #maemo | 16:09 | |
| *** eMHa has quit IRC | 16:37 | |
| *** eMHa has joined #maemo | 16:40 | |
| *** dafox has joined #maemo | 16:45 | |
| *** dafox has quit IRC | 16:52 | |
| *** shentey has joined #maemo | 18:28 | |
| *** shentey has quit IRC | 18:48 | |
| L29Ah | One more step | 18:55 |
| *** shentey has joined #maemo | 18:56 | |
| *** fuz_ has quit IRC | 19:04 | |
| *** shentey has quit IRC | 19:12 | |
| *** fuz_ has joined #maemo | 19:35 | |
| *** L29Ah has quit IRC | 19:46 | |
| *** lobito has quit IRC | 20:49 | |
| *** cyphase has quit IRC | 21:36 | |
| *** cyphase has joined #maemo | 21:41 | |
| *** Roth has joined #maemo | 22:05 | |
| *** louisdk has joined #maemo | 22:07 | |
| *** Roth_ has joined #maemo | 22:08 | |
| *** Roth has quit IRC | 22:11 | |
| *** Roth has joined #maemo | 22:14 | |
| *** Roth_ has quit IRC | 22:16 | |
| *** Roth_ has joined #maemo | 22:19 | |
| *** xes_ has joined #maemo | 22:20 | |
| *** Roth has quit IRC | 22:21 | |
| *** xes has quit IRC | 22:23 | |
| *** Roth has joined #maemo | 22:24 | |
| *** Roth_ has quit IRC | 22:27 | |
| *** Roth has quit IRC | 22:32 | |
| sunshavi | ~info sunshavi | 23:00 |
| *** RzR has joined #maemo | 23:25 | |
| *** sunshavi has quit IRC | 23:53 | |
Generated by irclog2html.py 2.15.1 by Marius Gedminas - find it at mg.pov.lt!