From py.thoulon at gmail.com  Mon Aug 22 00:00:27 2005
From: py.thoulon at gmail.com (Pyt)
Date: Mon, 22 Aug 2005 09:00:27 +0200
Subject: [ID3 Dev] Padding
In-Reply-To: <20050820195204.7145.qmail@web86907.mail.ukl.yahoo.com>
References: <20050820195050.53484.qmail@web86904.mail.ukl.yahoo.com>
	 <20050820195204.7145.qmail@web86907.mail.ukl.yahoo.com>
Message-ID: <e81081bd050822000013f793c1@mail.gmail.com>

Padding is the bunch of null chars that lie between the end of the last 
frame and the end of the tag (the tag size, as given in the tag header, 
includes padding). A null char where you would expect a frame ID indicates 
start of padding (unless you're past the end of the tag).
 Rgds,
Pyt.

 On 8/20/05, Ion Todirel <iontodirel at yahoo.co.uk> wrote: 
> 
>  
> 
> *Ion Todirel <iontodirel at yahoo.co.uk>* wrote: 
> 
> Please help me, how to detect padding? four zeroed chars?
> 
> ------------------------------
> To help you stay safe and secure online, we've developed the all new *Yahoo! 
> Security Centre*<http://us.rd.yahoo.com/mail/uk/taglines/default/security_centre/*http://uk.security.yahoo.com/>
> . 
> 
>  ------------------------------
> How much free photo storage do you get? Store your holiday snaps for FREE 
> with Yahoo! Photos. *Get Yahoo! Photos*<http://us.rd.yahoo.com/mail/uk/taglines/default/photos/*http://uk.photos.yahoo.com/> 
> 
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.id3.org/pipermail/id3v2/attachments/20050822/9b7dc128/attachment.html>

From py.thoulon at gmail.com  Tue Aug 23 23:37:31 2005
From: py.thoulon at gmail.com (Pyt)
Date: Wed, 24 Aug 2005 08:37:31 +0200
Subject: [ID3 Dev] Padding
In-Reply-To: <20050823210026.70185.qmail@web86903.mail.ukl.yahoo.com>
References: <e81081bd050822221652f359d0@mail.gmail.com>
	 <20050823210026.70185.qmail@web86903.mail.ukl.yahoo.com>
Message-ID: <e81081bd050823233729b7fe34@mail.gmail.com>

On 8/23/05, Ion Todirel <iontodirel at yahoo.co.uk> wrote: 
> 
>  Pyt please help me, i don't know if i understand good:
>  
 Ben fired faster than I could... :-)
 Cheers,
Pyt.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.id3.org/pipermail/id3v2/attachments/20050824/90fd4736/attachment.html>

From iontodirel at yahoo.co.uk  Wed Aug 24 03:02:00 2005
From: iontodirel at yahoo.co.uk (Ion Todirel)
Date: Wed, 24 Aug 2005 11:02:00 +0100 (BST)
Subject: [ID3 Dev] Padding
In-Reply-To: <20050824004533.GA31513@ayup.limey.net>
Message-ID: <20050824100201.50651.qmail@web86904.mail.ukl.yahoo.com>

Thanks.

Ben Bennett <fiji at ayup.limey.net> wrote:On Tue, Aug 23, 2005 at 11:12:42PM +0100, Ion Todirel wrote:
> thanks very much Ben, i stiil dont understand why footer and padding are mutually exclusive? if i have this situation, all sems to be OK?
> 
> 
> 
> 
> 

> 
> 
> 

The footer doesn't work as you have indicated here. The footer is at
the end of a tag so:

[The following is not a legal layout due to padding and footer]









Would be the right layout. Except that as soon as you hit padding it
means that there is padding for the rest of the tag size. In the case
above the footer doesn't make sense since you can't find it easily (it
is primarily for when the tag is appended).

So if you have a totally appended tag:

[Also not legal]









The footer now makes sense since you find it easily by scanning
backwards and then you can find the tag start and parse normally.
Except that there is no real need for padding since when you are so
close to the end of the file it is easy to rewrite the remainder to
grow or shrink the tag.


> and second problem:
> 
> here is a simple tag:
> 
> 
> <"frame ID" header><"frame ID" content>
> ...
> <"frame IDi" header><"frame IDi" content>
> ...
> <"frame IDn" header><"frame IDn" content> (last frame (normal frame))
> 
> 
(lets say 100%)
> 
> 
> 

I am not sure what you mean by 100%. If you mean 100% of the
remainder of the tag size, then okay, but padding always has to be
100% of the remainder. So lets call it 100 bytes.

> 
> Ben, way just not add two new frames like this?:
> 
> 
> <"frame ID" header><"frame ID" content>
> ...
> <"frame IDi" header><"frame IDi" content>
> ...
> <"frame IDn" header><"frame IDn" content>
> 
> <"frame IDn+1" content><"frame IDn+1" content> (added frame)
> 
> <"frame IDn+2" content><"frame IDn+2" content> (added frame) 
> 
(after adding two frames padding = padding - x bytes) 
> 
> 

If X = the size of the 2 new frames, then that is exactly what you
would do.

> but add like you say?:
> 
> 
> <"frame ID1" header>
> <"frame ID2" header>
> <"frame ID3" header><"frame ID3" content>
> ...
> <"frame IDi" header><"frame IDi" content>
> ...
> <"frame IDn" header><"frame IDn" content> (last frame (normal frame))
> 
> <"frame ID1" content> 
> <"frame ID2" content> 
> 
(same situation) 
> 
> 

I did not mean to imply this and have confused you. The previous case
is what you would do.

-ben

---------------------------------------------------------------------
To unsubscribe, e-mail: id3v2-unsubscribe at id3.org
For additional commands, e-mail: id3v2-help at id3.org



		
---------------------------------
How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos. Get Yahoo! Photos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.id3.org/pipermail/id3v2/attachments/20050824/2477d867/attachment.html>

From iontodirel at yahoo.co.uk  Sun Aug 28 07:05:08 2005
From: iontodirel at yahoo.co.uk (Ion Todirel)
Date: Sun, 28 Aug 2005 15:05:08 +0100 (BST)
Subject: [ID3 Dev] Syncsafe integers and 2.3/2.4
In-Reply-To: <20050828003147.GA23857@ayup.limey.net>
Message-ID: <20050828140508.15200.qmail@web86903.mail.ukl.yahoo.com>

Windows Media Player, has a SDK for developers, if Player can read tag, then SDK provide this functionality. If iTunes has SDK use it. Way iTunes does'nt support ID3v2?

Ben Bennett <fiji at ayup.limey.net> wrote:

On Sat, Aug 27, 2005 at 02:18:27PM +0100, Ion Todirel wrote:
> If you wont to understand how something works you mast broks it in
> parts. Play with Mirosoft Windows Media Player SDK 10 or with
> Windows Media Format 9.5 SDK (in .NET), check winamp, 

I understand how the spec is supposed to work and have tested the
behavior with a ton of tag writing things. The behavior is mixed.
iTunes is not the only broken implementation, but there are some that
gets it right.

> way iTunes is so important for you? its more important to support
> standards! 

Yes, but iTunes is a large portion of the market. And saying "we
don't support iTunes, because Apple can't RTFM" is not the best
option.

> When in some case something doesn't work its best to not do it, in
> this case heuristical algoritm, in some cases it will not work.

So. Better than blindly crashing all of the time. At least this way
it will work most of the time. And when it breaks I will add more
heuristics.

"Be pedantic about what yuou write and liberal about what you accept"

> Like i say, way iTunes?

Adressed above.

-ben

---------------------------------------------------------------------
To unsubscribe, e-mail: id3v2-unsubscribe at id3.org
For additional commands, e-mail: id3v2-help at id3.org



		
---------------------------------
How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos. Get Yahoo! Photos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.id3.org/pipermail/id3v2/attachments/20050828/fbc8706d/attachment.html>

From fiji at ayup.limey.net  Tue Aug 23 17:45:33 2005
From: fiji at ayup.limey.net (Ben Bennett)
Date: Tue, 23 Aug 2005 20:45:33 -0400
Subject: [ID3 Dev] Padding
In-Reply-To: <20050823221243.41855.qmail@web86908.mail.ukl.yahoo.com>
References: <20050823211031.GA26928@ayup.limey.net> <20050823221243.41855.qmail@web86908.mail.ukl.yahoo.com>
Message-ID: <20050824004533.GA31513@ayup.limey.net>

On Tue, Aug 23, 2005 at 11:12:42PM +0100, Ion Todirel wrote:
> thanks very much Ben, i stiil dont understand why footer and padding are mutually exclusive? if i have this situation, all sems to be OK?
>  
> <tag header>
> <TIT1 header><TIT1 contents>
> <TIT2 header><TIT2 contents>
> <padding>
> <mp3 audio data>
> <tag footer>
> <ID3v1 tag>

The footer doesn't work as you have indicated here.  The footer is at
the end of a tag so:

[The following is not a legal layout due to padding and footer]
<tag header>
<TIT1 header><TIT1 contents>
<TIT2 header><TIT2 contents>
<padding>
<tag footer>
<mp3 audio data>
<ID3v1 tag>

Would be the right layout.  Except that as soon as you hit padding it
means that there is padding for the rest of the tag size.  In the case
above the footer doesn't make sense since you can't find it easily (it
is primarily for when the tag is appended).

So if you have a totally appended tag:

[Also not legal]
<mp3 audio data>
<tag header>
<TIT1 header><TIT1 contents>
<TIT2 header><TIT2 contents>
<padding>
<tag footer>
<ID3v1 tag>

The footer now makes sense since you find it easily by scanning
backwards and then you can find the tag start and parse normally.
Except that there is no real need for padding since when you are so
close to the end of the file it is easy to rewrite the remainder to
grow or shrink the tag.
  

> and second problem:
>  
> here is a simple tag:
>  
> <tag header>
> <"frame ID" header><"frame ID" content>
> ...
> <"frame IDi" header><"frame IDi" content>
> ...
> <"frame IDn" header><"frame IDn" content> (last frame (normal frame))
> 
> <padding>                                                 (lets say 100%)
> <audio data>
> <ID3v1>
> 

I am not sure what you mean by 100%.  If you mean 100% of the
remainder of the tag size, then okay, but padding always has to be
100% of the remainder.  So lets call it 100 bytes.

>  
> Ben, way just not add two new frames like this?:
>  
> <tag header>
> <"frame ID" header><"frame ID" content>
> ...
> <"frame IDi" header><"frame IDi" content>
> ...
> <"frame IDn" header><"frame IDn" content>
> 
> <"frame IDn+1" content><"frame IDn+1" content> (added frame)
> 
> <"frame IDn+2" content><"frame IDn+2" content> (added frame)                                 
> <padding>                                                          (after adding two frames padding = padding - x bytes)   
> <audio data>
> <ID3v1>

If X = the size of the 2 new frames, then that is exactly what you
would do.

> but add like you say?:
>  
> <tag header>
> <"frame ID1" header>
> <"frame ID2" header>
> <"frame ID3" header><"frame ID3" content>
> ...
> <"frame IDi" header><"frame IDi" content>
> ...
> <"frame IDn" header><"frame IDn" content> (last frame (normal frame))
> 
> <"frame ID1" content>          
> <"frame ID2" content>                                  
> <padding>                                                 (same situation) 
> <audio data>
> <ID3v1>

I did not mean to imply this and have confused you.  The previous case
is what you would do.

		-ben

---------------------------------------------------------------------
To unsubscribe, e-mail: id3v2-unsubscribe at id3.org
For additional commands, e-mail: id3v2-help at id3.org


From gadigeppa.tuppad at flextronicssoftware.com  Sun Aug 28 22:01:51 2005
From: gadigeppa.tuppad at flextronicssoftware.com (gadigeppa.tuppad at flextronicssoftware.com)
Date: Mon, 29 Aug 2005 10:31:51 +0530
Subject: [ID3 Dev] Query regarding "iD3 tag"
Message-ID: <OF55EAC826.70E8DDBE-ON6525706C.001B889B-6525706C.001B379D@flextronicssoftware.com>






Hi,

I need to know in case Id3v2.4 tag is in the end of the file (as it is
optional to keep it in beginning or end), how does an mp3 decoder come to
know that it is reading tag information not audio data?
Please answer the same question in case of ID3v1 tag too. Do all the mp3
decoders look for id3v1 tag at last 128 bytes of the file? or they just hit
some data they can not understand and stop playing??

Thanks in anticipation!

Regards,
Gadigeppa



***********************  FSS-Unclassified   ***********************
"DISCLAIMER: This message is proprietary to Hughes Software Systems Limited
(HSS) and is intended solely for the use of the individual to whom it is
addressed. It may contain  privileged or confidential information and
should not be circulated or used for any purpose other than for what it is
intended. If you have received this message in error, please notify the
originator immediately. If you are not the intended recipient, you are
notified that you are strictly prohibited from using, copying, altering, or
disclosing the contents of this message. HSS accepts no responsibility for
loss or damage arising from the use of the information transmitted by this
email including damage from virus."


---------------------------------------------------------------------
To unsubscribe, e-mail: id3v2-unsubscribe at id3.org
For additional commands, e-mail: id3v2-help at id3.org


From fiji at ayup.limey.net  Tue Aug 23 14:10:31 2005
From: fiji at ayup.limey.net (Ben Bennett)
Date: Tue, 23 Aug 2005 17:10:31 -0400
Subject: [ID3 Dev] Padding
In-Reply-To: <20050823210026.70185.qmail@web86903.mail.ukl.yahoo.com>
References: <e81081bd050822221652f359d0@mail.gmail.com> <20050823210026.70185.qmail@web86903.mail.ukl.yahoo.com>
Message-ID: <20050823211031.GA26928@ayup.limey.net>

On Tue, Aug 23, 2005 at 10:00:26PM +0100, Ion Todirel wrote:
> If i use padding to add a new frame: let's say that i have a tag with TIT1 and TIT2: 
>  
> <tag header>
> <TIT1 header><TIT1 contents>
> <TIT2 header><TIT2 contents>
> <end of the tag, no padding>
> <mp3 audio data>
> <tag footer>
> <ID3v1 tag>

Yes.
 
>  
> if i add a new frame (TALB) using padding tag will look like this?:
>  
> <tag header>
> <TALB header only>
> <TIT1 header><TIT1 contents>
> <TIT2 header><TIT2 contents>
> <end of the tag, start of the padding>
> <TALB contents (or TALB enlargement, if padding is used to enlarge a frame)>
> <mp3 audio data>
> <no footer>
> <ID3v1 tag>

No.

The TALB is a frame and MUST be inside the ID3 tag.  The padding also
falls inside the tag, (i.e. is included in the tag size) but MUST
follow all frames.  So it will look like:

<tag header>
<TALB header only>
<TIT1 header><TIT1 contents>
<TIT2 header><TIT2 contents>
<TALB contents (or TALB enlargement, if padding is used to enlarge a frame)>
<padding>
<end of the tag>
<mp3 audio data>
<no footer>
<ID3v1 tag>

> if frame TALB already exists in the tag:
>  
> <tag header>
> <TALB header><TALB contents>
> 
> <TIT1 header><TIT1 contents>
> <TIT2 header><TIT2 contents>
> <end of the tag, no padding>
> <mp3 audio data>
> <tag footer>
> <ID3v1 tag>

Okay.

> and i extend it using padding:
>  
> <tag header>
> <TALB header><TALB contents>
> 
> <TIT1 header><TIT1 contents>
> <TIT2 header><TIT2 contents>
> <end of the tag, start of the padding>
> <TALB aditional content (or TALB enlargement)>
> <mp3 audio data>
> <no footer>
> <ID3v1 tag>

Again, no.  You can not split the additional content from TALB.  There
can only be one TALB frame per tag.  So it will look like:

<tag header>
<TALB header><New TALB contents>
<TIT1 header><TIT1 contents>
<TIT2 header><TIT2 contents>
<padding>
<end of the tag>
<mp3 audio data>
<no footer>
<ID3v1 tag>

  
> But how to detect when padding starts? just before last frame, but how to detect when is last frame?

Padding falls after all frames.  It is the last thing in the tag.  So
if you expect to see a frame header but read all nuls then you are
into the padding.

The only point to padding is to prevent rewriting all of the content
in the file if a tag grows.  So if you have N bytes of padding at the
end of a frame and you grow the content by N bytes or less, you can
just rewrite the tag and shrink the padding.  But if you need to grow
something by more than N bytes then you need to fiddle with the
remainder of the file as well.

Does that help?

		      -ben

---------------------------------------------------------------------
To unsubscribe, e-mail: id3v2-unsubscribe at id3.org
For additional commands, e-mail: id3v2-help at id3.org


From fiji at ayup.limey.net  Wed Aug 24 06:57:00 2005
From: fiji at ayup.limey.net (Ben Bennett)
Date: Wed, 24 Aug 2005 09:57:00 -0400
Subject: [ID3 Dev] ID3v2.4.0 native frames
In-Reply-To: <20050824124219.45106.qmail@web86903.mail.ukl.yahoo.com>
References: <20050824124219.45106.qmail@web86903.mail.ukl.yahoo.com>
Message-ID: <20050824135700.GA30106@ayup.limey.net>

On Wed, Aug 24, 2005 at 01:42:19PM +0100, Ion Todirel wrote:
> Ben, look on this frame (or others):
> 
> <COMM>
> Text encoding              $xx
> Language                    $xx xx xx
> Short content descrip.  <text string according to encoding> $00 (00)
> The actual text             <full text string according to encoding>
>  
> $00 is a byte (new byte(); (0)) separator between "Short content descrip." and "Text"?

The reason the terminator is specified as $00 (00) is that the
terminator may be one byte or it may be two depending on the encoding.
Section 4 of the "Main Structure" document
(http://www.id3.org/id3v2.4.0-structure.txt) outlines the rules:
     $00   ISO-8859-1 [ISO-8859-1]. Terminated with $00.
     $01   UTF-16 [UTF-16] encoded Unicode [UNICODE] with BOM. All
           strings in the same frame SHALL have the same byteorder.
           Terminated with $00 00.
     $02   UTF-16BE [UTF-16] encoded Unicode [UNICODE] without BOM.
           Terminated with $00 00.
     $03   UTF-8 [UTF-8] encoded Unicode [UNICODE]. Terminated with $00.

So for encodings $00 and $03 the terminator is $00, and for $01 and
$02 (the wide character Unicode encodings) the terminator is $00 $00.
 
> For some frames is not specified String Encoding, for example for frame "WXXX":
> 
> <"WXXX">
> Text encoding     $xx
> Description         <text string according to encoding> $00 (00)
> URL                   <text string>
> 
> URL is Encoded with Latin1Encoding ("ISO-8859-1") ?
> 
> or for "UFID" frame, what encoding shuld use to decode "Identifier" from byte to string? "ISO-8859-1" ?

 From the same section of the spec:
  If nothing else is said, strings, including numeric strings and URLs
  [URL], are represented as ISO-8859-1 [ISO-8859-1] characters in the
  range $20 - $FF. Such strings are represented in frame descriptions
  as <text string>, or <full text string> if newlines are allowed. If
  nothing else is said newline character is forbidden. In ISO-8859-1 a
  newline is represented, when allowed, with $0A only.

Note that in the WXXX case it should be URL encoded anyway.

Also note the rules about newlines and the distinction in the spec
between <text string> and <full text string>.

The spec is unlear about what to do with newlines when the encoding is
not ISO-8859-1.

			       -ben

---------------------------------------------------------------------
To unsubscribe, e-mail: id3v2-unsubscribe at id3.org
For additional commands, e-mail: id3v2-help at id3.org


From fiji at ayup.limey.net  Mon Aug 22 06:40:16 2005
From: fiji at ayup.limey.net (Ben Bennett)
Date: Mon, 22 Aug 2005 09:40:16 -0400
Subject: [ID3 Dev] Syncsafe integers and 2.3/2.4
In-Reply-To: <20050822044051.70259.qmail@web86910.mail.ukl.yahoo.com>
References: <20050821173852.n8nisjr5q7pcsk88@webmail.mit.edu> <20050822044051.70259.qmail@web86910.mail.ukl.yahoo.com>
Message-ID: <20050822134016.GA12114@ayup.limey.net>

On Mon, Aug 22, 2005 at 05:40:51AM +0100, Ion Todirel wrote:
> To Ben Bennett:
>  
> Both 2.3 and 2.4 use Syncsafe integers. ID3v2.4 its a nice one, i
> recommend you to implement yourself ID3v2.4/Reader-Writer and threat
> there your syncsafe problems, also if you you have problems with
> syncsafe email me, i develop some nice methods.

Strictly speaking both do.  But they use them in different places.

Look at the document detailing the changes from 2.3 to 2.4 [1].  It
explicitly says:
   Unsynchronisation [S:6.1] is done on frame level, instead of on tag
   level, making it easier to skip frames, increasing thestreamability
   of the tag. The unsynchronisation flag in the header [S:3.1]
   indicates if all frames has been unsynchronized, while the new
   unsynchronisation flag in the frame header [S:4.1.2] indicates
   unsynchronisation. To avoid false synchronisations in the frame
*  header the size description and flag field has been rewritten [S:4].
   Resynchronisation of the complete tag when the unsynchronisation flag
   in the tag header is set might result in a corrupt tag.

The line marked with * is the change that we are discussing.  Prior to
2.4 the frame size was a 32 bit integer.  In 2.4 it is a 28 bit
syncsafe integer.  However, if you generate a large frame in iTunes
and save the tag as 2.4 you will see that the frame size is not syncsafe.

Please tell me how you reliably determine the frame size in 2.4.  I
have a bunch of heuristics that are inefficient and annoying:
 - Look at the bit pattern to see if it has 1s where a true syncsafe
   integer ought to have 0s
 - Check to make sure we are at the end of the tag (or in the
   padding), or at the start of the next frame.  And if not, treat it as
   a non-syncsafe int and see if it is better

I have excahnged email with the author of the Godfather tagging
program and he only supports reading of 2.4 tags.  He will only write
to the 2.3 because the 2.4 implementation is too messed up by various
people.

Which is too bad since 2.4 supports UTF-8 which I want.  I think it
would be cool to release a 2.3.1 version that supports the UTF-8
encoding.  And I think htere needs to be a 2.4.1 release that just
updates unclear portions of the 2.4 spec and makes sure that people
know that frame sizes are sync-safe now.

			-ben

[1] http://www.id3.org/id3v2.4.0-changes.txt

---------------------------------------------------------------------
To unsubscribe, e-mail: id3v2-unsubscribe at id3.org
For additional commands, e-mail: id3v2-help at id3.org


From fiji at ayup.limey.net  Sun Aug 21 17:10:39 2005
From: fiji at ayup.limey.net (Ben Bennett)
Date: Sun, 21 Aug 2005 20:10:39 -0400
Subject: [ID3 Dev] Syncsafe integers and 2.3/2.4
In-Reply-To: <20050821173852.n8nisjr5q7pcsk88@webmail.mit.edu>
References: <20050821173852.n8nisjr5q7pcsk88@webmail.mit.edu>
Message-ID: <20050822001039.GA15058@ayup.limey.net>

On Sun, Aug 21, 2005 at 05:38:52PM -0400, Robert Crowell wrote:
> Is iTunes at fault for not converting the USER frame's size to a synchsafe
> integer when converting a tag to 2.4?  Am I mis-reading the ID3v2
> specification?  If it turns out that iTunes is doing something silly, how would
> I detect it in my code?

Sadly iTunes has totally buggered up their implementation of 2.4.
They do not honor the syncsafe integer rules so reading a 2.4 frame is
sketchy.  The best you can do is look at the syncsafe integer and see
if it has bits set where they are not allowed to be set, and if they
are then treat it as a non-syncsafe int.

Unfortunately if a frame is already syncsafe and imported by something
else, iTunes leaves it alone until it needs to write it.  Gah!

> P.S. I have also tested the tags generated by the Media Rage application for
> Mac.  The behavior is the same.  What am I doing wrong?

Quite possibly it was written by the same library (Quicktime).

There is a PHP ID3 library that has decent support for dealing with
this.  I looked at their code when I was wrestling with this issue.
They have comments indicating iTunes is borked too.

I would avoid ID3v2.4.

		      -ben

---------------------------------------------------------------------
To unsubscribe, e-mail: id3v2-unsubscribe at id3.org
For additional commands, e-mail: id3v2-help at id3.org


From fiji at ayup.limey.net  Sat Aug 27 17:31:47 2005
From: fiji at ayup.limey.net (Ben Bennett)
Date: Sat, 27 Aug 2005 20:31:47 -0400
Subject: [ID3 Dev] Syncsafe integers and 2.3/2.4
In-Reply-To: <20050827131827.34767.qmail@web86902.mail.ukl.yahoo.com>
References: <20050827033626.GA22938@ayup.limey.net> <20050827131827.34767.qmail@web86902.mail.ukl.yahoo.com>
Message-ID: <20050828003147.GA23857@ayup.limey.net>



On Sat, Aug 27, 2005 at 02:18:27PM +0100, Ion Todirel wrote:
> If you wont to understand how something works you mast broks it in
> parts. Play with Mirosoft Windows Media Player SDK 10 or with
> Windows Media Format 9.5 SDK (in .NET), check winamp, 

I understand how the spec is supposed to work and have tested the
behavior with a ton of tag writing things.  The behavior is mixed.
iTunes is not the only broken implementation, but there are some that
gets it right.

> way iTunes is so important for you? its more important to support
> standards! 

Yes, but iTunes is a large portion of the market.  And saying "we
don't support iTunes, because Apple can't RTFM" is not the best
option.

> When in some case something doesn't work its best to not do it, in
> this case heuristical algoritm, in some cases it will not work.

So.  Better than blindly crashing all of the time.  At least this way
it will work most of the time.  And when it breaks I will add more
heuristics.

"Be pedantic about what yuou write and liberal about what you accept"

> Like i say, way iTunes?

Adressed above.

	      -ben

---------------------------------------------------------------------
To unsubscribe, e-mail: id3v2-unsubscribe at id3.org
For additional commands, e-mail: id3v2-help at id3.org


From iontodirel at yahoo.co.uk  Mon Aug 22 07:00:37 2005
From: iontodirel at yahoo.co.uk (Ion Todirel)
Date: Mon, 22 Aug 2005 15:00:37 +0100 (BST)
Subject: [ID3 Dev] Syncsafe integers and 2.3/2.4
In-Reply-To: <20050822134016.GA12114@ayup.limey.net>
Message-ID: <20050822140037.95547.qmail@web86902.mail.ukl.yahoo.com>

Slow, please
 
First you cannot determine if is syncsafe or not! based on MSB in all bytes. Second in your library you only can put an option to read frame size as syncsafe or as normal integers, you cannot read ID3v2.3 or ID3v2.4 as normal because iTunes write these information as normal integers, look how WindowsMedia Player or Winamp write  these, if what you say is true then iTunes does'n support ID3v2 standard.

Ben Bennett <fiji at ayup.limey.net> wrote:
On Mon, Aug 22, 2005 at 05:40:51AM +0100, Ion Todirel wrote:
> To Ben Bennett:
> 
> Both 2.3 and 2.4 use Syncsafe integers. ID3v2.4 its a nice one, i
> recommend you to implement yourself ID3v2.4/Reader-Writer and threat
> there your syncsafe problems, also if you you have problems with
> syncsafe email me, i develop some nice methods.

Strictly speaking both do. But they use them in different places.

Look at the document detailing the changes from 2.3 to 2.4 [1]. It
explicitly says:
Unsynchronisation [S:6.1] is done on frame level, instead of on tag
level, making it easier to skip frames, increasing thestreamability
of the tag. The unsynchronisation flag in the header [S:3.1]
indicates if all frames has been unsynchronized, while the new
unsynchronisation flag in the frame header [S:4.1.2] indicates
unsynchronisation. To avoid false synchronisations in the frame
* header the size description and flag field has been rewritten [S:4].
Resynchronisation of the complete tag when the unsynchronisation flag
in the tag header is set might result in a corrupt tag.

The line marked with * is the change that we are discussing. Prior to
2.4 the frame size was a 32 bit integer. In 2.4 it is a 28 bit
syncsafe integer. However, if you generate a large frame in iTunes
and save the tag as 2.4 you will see that the frame size is not syncsafe.

Please tell me how you reliably determine the frame size in 2.4. I
have a bunch of heuristics that are inefficient and annoying:
- Look at the bit pattern to see if it has 1s where a true syncsafe
integer ought to have 0s
- Check to make sure we are at the end of the tag (or in the
padding), or at the start of the next frame. And if not, treat it as
a non-syncsafe int and see if it is better

I have excahnged email with the author of the Godfather tagging
program and he only supports reading of 2.4 tags. He will only write
to the 2.3 because the 2.4 implementation is too messed up by various
people.

Which is too bad since 2.4 supports UTF-8 which I want. I think it
would be cool to release a 2.3.1 version that supports the UTF-8
encoding. And I think htere needs to be a 2.4.1 release that just
updates unclear portions of the 2.4 spec and makes sure that people
know that frame sizes are sync-safe now.

-ben

[1] http://www.id3.org/id3v2.4.0-changes.txt

---------------------------------------------------------------------
To unsubscribe, e-mail: id3v2-unsubscribe at id3.org
For additional commands, e-mail: id3v2-help at id3.org


		
---------------------------------
Yahoo! Messenger  NEW - crystal clear PC to PC calling worldwide with voicemail 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.id3.org/pipermail/id3v2/attachments/20050822/076b48c8/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Class1.cs
Type: application/octet-stream
Size: 3261 bytes
Desc: 1434560400-Class1.cs
URL: <http://lists.id3.org/pipermail/id3v2/attachments/20050822/076b48c8/attachment.obj>
-------------- next part --------------
---------------------------------------------------------------------
To unsubscribe, e-mail: id3v2-unsubscribe at id3.org
For additional commands, e-mail: id3v2-help at id3.org

From iontodirel at yahoo.co.uk  Tue Aug 23 14:00:26 2005
From: iontodirel at yahoo.co.uk (Ion Todirel)
Date: Tue, 23 Aug 2005 22:00:26 +0100 (BST)
Subject: [ID3 Dev] Padding
In-Reply-To: <e81081bd050822221652f359d0@mail.gmail.com>
Message-ID: <20050823210026.70185.qmail@web86903.mail.ukl.yahoo.com>

Pyt please help me, i don't know if i understand good:
 
If i use padding to add a new frame: let's say that i have a tag with TIT1 and TIT2: 
 
<tag header>
<TIT1 header><TIT1 contents>
<TIT2 header><TIT2 contents>
<end of the tag, no padding>
<mp3 audio data>
<tag footer>
<ID3v1 tag>

 
if i add a new frame (TALB) using padding tag will look like this?:
 
<tag header>
<TALB header only>
<TIT1 header><TIT1 contents>
<TIT2 header><TIT2 contents>
<end of the tag, start of the padding>
<TALB contents (or TALB enlargement, if padding is used to enlarge a frame)>
<mp3 audio data>
<no footer>
<ID3v1 tag>

 
if frame TALB already exists in the tag:
 
<tag header>
<TALB header><TALB contents>

<TIT1 header><TIT1 contents>
<TIT2 header><TIT2 contents>
<end of the tag, no padding>
<mp3 audio data>
<tag footer>
<ID3v1 tag>


 
and i extend it using padding:
 
<tag header>
<TALB header><TALB contents>

<TIT1 header><TIT1 contents>
<TIT2 header><TIT2 contents>
<end of the tag, start of the padding>
<TALB aditional content (or TALB enlargement)>
<mp3 audio data>
<no footer>
<ID3v1 tag>
 
But how to detect when padding starts? just before last frame, but how to detect when is last frame?
 
if i have two frame, with its enlargmens in padding:
 
<tag header>
<TALB header><TALB contents>

<TIT1 header><TIT1 contents>
<TIT2 header><TIT2 contents>
<end of the tag, start of the padding>
<$00 00 00 00 (start of padding?) TALB aditional content>
<$00 00 00 00 TIT1 aditional content>

<mp3 audio data>
<no footer>
<ID3v1 tag>
 
in both cases if i use padding to enlarge or add new frames information in padding is just bytes, no headers or flags, before data content?
 
Thanks, very mach! and help me please :(

Pyt <py.thoulon at gmail.com> wrote:




>Tag size included on tag header, dont include padding? 
 
The tag size in the tag header includes the size of : (1) the extended header, (2) the actual frames, (3) the padding. See id3v2.4.0-structure.txt, section 3.1.
 
 >If padding is used to add aditional frames, where is the frame ID since ID is four null chars? or if padding is used to enlarge existing frames how to know what frame are enlarged?
 
Let's say you have a tag with padding so that the total size of header+extended header+frames+padding = 4K (a typical case); the size in the tag header would be 4086 (4096 - size of header, which is ten bytes). Let's also say the actual frames use up only 2000 of those 4086 bytes. That leaves 2086 bytes of padding. 
Now, if you want to add a new frame to the tag, you can stick it right after the last of the existing frames, overwriting the padding bytes with the new frame contents. You would make no other change to your tag: the size remains the same, only the relative sizes of frames vs. padding have changed; if you new frame is say 50 bytes long, the tag size still indicates 4086, the total frame size is now 2050, and the padding size only 2036. 
The advantage of this is that if you have enough padding, you can play with the frames without having to rewrite the whole MP3 file (which is generally a lengthy exercise, given the sizes of these beast).
 
Enlarging a frame means a bit more extensive work because frames have to remain in one piece. Let's say you have a tag with TIT1, TALB, TIPL, TIT2 frames in this order, followed by padding. If you want to enlarge TIPL for example, you will need to rewrite your whole tag (or at least the end of it, starting with TIPL) over the original one, eating up into the padding. 
 
>And if footer is writting to end of file, is'n possible to write an ID3v1 Tag, right? since ID3v1 tag is located from 0 to 128 bytes at end of file.
Yes, you can. The ID3v2 footer should be located just before the ID3v1 tag. A good parsing algorithm looking for an ID3v2 footer should be able to skip the v1 tag.
 
Regards,
Pyt.


Pyt <py.thoulon at gmail.com> wrote:>Tag size included on tag header, dont include padding? 
 
The tag size in the tag header includes the size of : (1) the extended header, (2) the actual frames, (3) the padding. See id3v2.4.0-structure.txt, section 3.1.
 
 >If padding is used to add aditional frames, where is the frame ID since ID is four null chars? or if padding is used to enlarge existing frames how to know what frame are enlarged?
 
Let's say you have a tag with padding so that the total size of header+extended header+frames+padding = 4K (a typical case); the size in the tag header would be 4086 (4096 - size of header, which is ten bytes). Let's also say the actual frames use up only 2000 of those 4086 bytes. That leaves 2086 bytes of padding. 
Now, if you want to add a new frame to the tag, you can stick it right after the last of the existing frames, overwriting the padding bytes with the new frame contents. You would make no other change to your tag: the size remains the same, only the relative sizes of frames vs. padding have changed; if you new frame is say 50 bytes long, the tag size still indicates 4086, the total frame size is now 2050, and the padding size only 2036. 
The advantage of this is that if you have enough padding, you can play with the frames without having to rewrite the whole MP3 file (which is generally a lengthy exercise, given the sizes of these beast).
 
Enlarging a frame means a bit more extensive work because frames have to remain in one piece. Let's say you have a tag with TIT1, TALB, TIPL, TIT2 frames in this order, followed by padding. If you want to enlarge TIPL for example, you will need to rewrite your whole tag (or at least the end of it, starting with TIPL) over the original one, eating up into the padding. 
 
>And if footer is writting to end of file, is'n possible to write an ID3v1 Tag, right? since ID3v1 tag is located from 0 to 128 bytes at end of file.
Yes, you can. The ID3v2 footer should be located just before the ID3v1 tag. A good parsing algorithm looking for an ID3v2 footer should be able to skip the v1 tag.
 
Regards,
Pyt.


		
---------------------------------
To help you stay safe and secure online, we've developed the all new Yahoo! Security Centre.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.id3.org/pipermail/id3v2/attachments/20050823/f5764127/attachment.html>

From iontodirel at yahoo.co.uk  Wed Aug 24 07:17:10 2005
From: iontodirel at yahoo.co.uk (Ion Todirel)
Date: Wed, 24 Aug 2005 15:17:10 +0100 (BST)
Subject: [ID3 Dev] ID3v2.4.0 native frames
In-Reply-To: <20050824135700.GA30106@ayup.limey.net>
Message-ID: <20050824141710.38668.qmail@web86901.mail.ukl.yahoo.com>

Thanks again for help, 
 
John.

Ben Bennett <fiji at ayup.limey.net> wrote:
On Wed, Aug 24, 2005 at 01:42:19PM +0100, Ion Todirel wrote:
> Ben, look on this frame (or others):
> 
> 
> Text encoding $xx
> Language $xx xx xx
> Short content descrip. $00 (00)
> The actual text 
> 
> $00 is a byte (new byte(); (0)) separator between "Short content descrip." and "Text"?

The reason the terminator is specified as $00 (00) is that the
terminator may be one byte or it may be two depending on the encoding.
Section 4 of the "Main Structure" document
(http://www.id3.org/id3v2.4.0-structure.txt) outlines the rules:
$00 ISO-8859-1 [ISO-8859-1]. Terminated with $00.
$01 UTF-16 [UTF-16] encoded Unicode [UNICODE] with BOM. All
strings in the same frame SHALL have the same byteorder.
Terminated with $00 00.
$02 UTF-16BE [UTF-16] encoded Unicode [UNICODE] without BOM.
Terminated with $00 00.
$03 UTF-8 [UTF-8] encoded Unicode [UNICODE]. Terminated with $00.

So for encodings $00 and $03 the terminator is $00, and for $01 and
$02 (the wide character Unicode encodings) the terminator is $00 $00.

> For some frames is not specified String Encoding, for example for frame "WXXX":
> 
> <"WXXX">
> Text encoding $xx
> Description $00 (00)
> URL 
> 
> URL is Encoded with Latin1Encoding ("ISO-8859-1") ?
> 
> or for "UFID" frame, what encoding shuld use to decode "Identifier" from byte to string? "ISO-8859-1" ?

 From the same section of the spec:
If nothing else is said, strings, including numeric strings and URLs
[URL], are represented as ISO-8859-1 [ISO-8859-1] characters in the
range $20 - $FF. Such strings are represented in frame descriptions
as , or if newlines are allowed. If
nothing else is said newline character is forbidden. In ISO-8859-1 a
newline is represented, when allowed, with $0A only.

Note that in the WXXX case it should be URL encoded anyway.

Also note the rules about newlines and the distinction in the spec
between and .

The spec is unlear about what to do with newlines when the encoding is
not ISO-8859-1.

-ben

---------------------------------------------------------------------
To unsubscribe, e-mail: id3v2-unsubscribe at id3.org
For additional commands, e-mail: id3v2-help at id3.org


		
---------------------------------
Yahoo! Messenger  NEW - crystal clear PC to PC calling worldwide with voicemail 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.id3.org/pipermail/id3v2/attachments/20050824/d9164655/attachment.html>

From iontodirel at yahoo.co.uk  Wed Aug 24 05:42:19 2005
From: iontodirel at yahoo.co.uk (Ion Todirel)
Date: Wed, 24 Aug 2005 13:42:19 +0100 (BST)
Subject: [ID3 Dev] ID3v2.4.0 native frames
Message-ID: <20050824124219.45106.qmail@web86903.mail.ukl.yahoo.com>

Ben, look on this frame (or others):

<COMM>
Text encoding              $xx
Language                    $xx xx xx
Short content descrip.  <text string according to encoding> $00 (00)
The actual text             <full text string according to encoding>
 
$00 is a byte (new byte(); (0)) separator between "Short content descrip." and "Text"?


For some frames is not specified String Encoding, for example for frame "WXXX":

<"WXXX">
Text encoding     $xx
Description         <text string according to encoding> $00 (00)
URL                   <text string>

URL is Encoded with Latin1Encoding ("ISO-8859-1") ?

or for "UFID" frame, what encoding shuld use to decode "Identifier" from byte to string? "ISO-8859-1" ?

<"UFID">
Owner identifier    <text string> $00
Identifier              <up to 64 bytes binary data>


 
 

		
---------------------------------
How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos. Get Yahoo! Photos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.id3.org/pipermail/id3v2/attachments/20050824/ba183ea6/attachment.html>

From iontodirel at yahoo.co.uk  Thu Aug 25 12:43:09 2005
From: iontodirel at yahoo.co.uk (Ion Todirel)
Date: Thu, 25 Aug 2005 20:43:09 +0100 (BST)
Subject: [ID3 Dev] ID3v2.4.0 Frames
Message-ID: <20050825194309.38753.qmail@web86902.mail.ukl.yahoo.com>

Hi Ben, i have some more questions about the frames, if you could help me...
 
In Frame Header:
<Frame ID (4Bytes)>
<Size     (4Bytes)>
<Flags    (2Bytes)>  
---------------------
If in frame format flags is set: grouping identity, compression and encryption (with "1")
then, frame will look like that?:
---------------------
<group identifier byte (1Byte)>
<Data Length Indicator byte (1Byte)> (compression)
<Encryption method byte (1Byte)      (encryption) 
---------------------
Data lenght indicator value is stored like normal decimal number? if Data lenght Indicator flag is set (last bit in frame format flags) what this means?
 
Encription require a ENCR frame, right? but grouping identity require a GRID frame? or SIGN frame? if not for what is
SIGN and GRID frames, if not for working with grouping identity byte data.
---------------------
<Data lenght indicator (4Bytes)
<frame contents>
---------------------
she have same value like "Frame size"?, if all flags is zeroed (status and format flags)
is required to add a Data Lenght Indicator? why?
 
And what is the point of Unsyncronysation and what is that?, "The only purpose of unsynchronisation is to make the ID3v2 tag as
compatible as possible with existing software and hardware...", and Syncsafe. Its easy to make algorithms
to decode and encode synsafe integers, but way, what is the purpose of that, i dont find any explanation
on id3.org of that. On mp3-files, "unsyncronisation: is used?
 
About frames, in ID3v2.40 main structure its said that a frame must have at least one byte lenght
excluding header. Theoretically if i have these situation
----------------------
<TIT1>
<$00 00 00 01>
<$xx xx>
<Encoding $xx>
null             (here must be Information, but since size of the frame is one byte lenght, all its OK)
----------------------
i must report a error that Information must be at least 1Byte lenght (minimum size of TIT1 must be 2Bytes)
right? what is the sense of encoding if text that must be encoded isnt present?
 
CRC is equal with Tag size - extended Header, if all is OK?
 
and finally least question, if a frame have  Tag alter preservation or File alter preservation flags set, leave this frame
and go to next?
 
Thanks, hope you help me.

		
---------------------------------
How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos. Get Yahoo! Photos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.id3.org/pipermail/id3v2/attachments/20050825/c10ca4db/attachment.html>

From fiji at ayup.limey.net  Mon Aug 22 10:31:43 2005
From: fiji at ayup.limey.net (Ben Bennett)
Date: Mon, 22 Aug 2005 13:31:43 -0400
Subject: [ID3 Dev] Padding
In-Reply-To: <20050822135039.91270.qmail@web86910.mail.ukl.yahoo.com>
References: <e81081bd050822000013f793c1@mail.gmail.com> <20050822135039.91270.qmail@web86910.mail.ukl.yahoo.com>
Message-ID: <20050822173143.GA22931@ayup.limey.net>

Also the Programming Guides (http://www.id3.org/id3guide.htm)
recommend padding out to a full block on whatever system is doing the
writing (see 3.1 Padding).

		 -ben

---------------------------------------------------------------------
To unsubscribe, e-mail: id3v2-unsubscribe at id3.org
For additional commands, e-mail: id3v2-help at id3.org


From iontodirel at yahoo.co.uk  Sun Aug 28 08:51:30 2005
From: iontodirel at yahoo.co.uk (Ion Todirel)
Date: Sun, 28 Aug 2005 16:51:30 +0100 (BST)
Subject: [ID3 Dev] iTunes
Message-ID: <20050828155130.20178.qmail@web86908.mail.ukl.yahoo.com>

I that, that iTunes is a little smarter, Windows Media Player has much more support for ID3 than iTunes, thats way iTunes does'nt support ID3.
 
 

                                                        WMP 10 - Lyrics Support
WMP 10 - Full 
ID3 Support


You may ask Apple's developers http://www.apple.com/support/itunes/windows about iTunes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.id3.org/pipermail/id3v2/attachments/20050828/28c6cf2f/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 10293 bytes
Desc: not available
URL: <http://lists.id3.org/pipermail/id3v2/attachments/20050828/28c6cf2f/attachment.jpe>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 10132 bytes
Desc: not available
URL: <http://lists.id3.org/pipermail/id3v2/attachments/20050828/28c6cf2f/attachment-0001.jpe>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 695 bytes
Desc: not available
URL: <http://lists.id3.org/pipermail/id3v2/attachments/20050828/28c6cf2f/attachment.gif>

From py.thoulon at gmail.com  Mon Aug 22 22:16:45 2005
From: py.thoulon at gmail.com (Pyt)
Date: Tue, 23 Aug 2005 07:16:45 +0200
Subject: [ID3 Dev] Padding
In-Reply-To: <20050822135039.91270.qmail@web86910.mail.ukl.yahoo.com>
References: <e81081bd050822000013f793c1@mail.gmail.com>
	 <20050822135039.91270.qmail@web86910.mail.ukl.yahoo.com>
Message-ID: <e81081bd050822221652f359d0@mail.gmail.com>

>Tag size included on tag header, dont include padding? 
 The tag size in the tag header includes the size of : (1) the extended 
header, (2) the actual frames, (3) the padding. See id3v2.4.0-structure.txt, 
section 3.1.
  >If padding is used to add aditional frames, where is the frame ID since 
ID is four null chars? or if padding is used to enlarge existing frames how 
to know what frame are enlarged?
 Let's say you have a tag with padding so that the total size of 
header+extended header+frames+padding = 4K (a typical case); the size in the 
tag header would be 4086 (4096 - size of header, which is ten bytes). Let's 
also say the actual frames use up only 2000 of those 4086 bytes. That leaves 
2086 bytes of padding.
Now, if you want to add a new frame to the tag, you can stick it right after 
the last of the existing frames, overwriting the padding bytes with the new 
frame contents. You would make no other change to your tag: the size remains 
the same, only the relative sizes of frames vs. padding have changed; if you 
new frame is say 50 bytes long, the tag size still indicates 4086, the total 
frame size is now 2050, and the padding size only 2036.
The advantage of this is that if you have enough padding, you can play with 
the frames without having to rewrite the whole MP3 file (which is generally 
a lengthy exercise, given the sizes of these beast).
 Enlarging a frame means a bit more extensive work because frames have to 
remain in one piece. Let's say you have a tag with TIT1, TALB, TIPL, TIT2 
frames in this order, followed by padding. If you want to enlarge TIPL for 
example, you will need to rewrite your whole tag (or at least the end of it, 
starting with TIPL) over the original one, eating up into the padding.
 >And if footer is writting to end of file, is'n possible to write an ID3v1 
Tag, right? since ID3v1 tag is located from 0 to 128 bytes at end of file.
Yes, you can. The ID3v2 footer should be located just before the ID3v1 tag. 
A good parsing algorithm looking for an ID3v2 footer should be able to skip 
the v1 tag.
 Regards,
Pyt.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.id3.org/pipermail/id3v2/attachments/20050823/864b57fa/attachment.html>

From crowell at MIT.EDU  Sun Aug 21 14:38:52 2005
From: crowell at MIT.EDU (Robert Crowell)
Date: Sun, 21 Aug 2005 17:38:52 -0400
Subject: [ID3 Dev] Syncsafe integers and 2.3/2.4
Message-ID: <20050821173852.n8nisjr5q7pcsk88@webmail.mit.edu>

I am working on a python library to read (and possibly write) id3v2 tags.

Reading over the specification several times, I wasn't able to determine whether
or not id3v2.3 frames use the 'synchsafe integers' that 2.4 frames do. (I
understand that both 2.3 and 2.4 use synchsafe ints for the _tag_ size, but I
couldn't determine what the _frame_ behavior is.)

While the specification seems to indicate that v2.3 frames DO NOT use synchsafe
ints and v2.4 frames DO use synchsafe integers, I am encountering some odd
behavior.  Using the 'convert ID3 tags' feature in iTunes 4.9, I convert a v2.3
tag to v2.4, but the frame size bytes don't seem to be any different in the 2.3
and 2.4 versions.  There is a long USER tag in one of my mp3 files (that was
created by an unknown application) that contains 370 ASCII characters.

The 4 size bytes in this frame header seem to be encoding this value as a
NON-synchsafe integer.  In my code, when 2.3 is detected it expects
non-synchsafe integers for frame sizes and everything is fine.  However, after
converting this tag to 2.4 in iTunes, it interprets the size as a synchsafe
integer, and because iTunes did not modify the size bytes when converting from
2.3 to 2.4, it gets the frame size wrong.  It decides the frame is only 242
bytes long rather than the correct 370.  (size:0,0,1,114 -> 370 normal, 242
synchsafe)

This causes a huge problem because not only is the USER tag not read completely,
but my code mistakenly interprets the next 10 bytes as a new frame header. 
Because the size portion of this header is clearly invalid (since this isn't a
real header at all), my code believes the size of the next frame is huge,
interpreting 'manc' as the frame size bytes.

size bytes: $00 00 01 72 (for both 2.3 and 2.4)
2.3 size: 370 (correct, assuming non-synchsafe int)
2.4 size: 242 (incorrect, assuming synchsafe int)

Is iTunes at fault for not converting the USER frame's size to a synchsafe
integer when converting a tag to 2.4?  Am I mis-reading the ID3v2
specification?  If it turns out that iTunes is doing something silly, how would
I detect it in my code?

Thanks for your help.

-Rob Crowell

P.S. I have also tested the tags generated by the Media Rage application for
Mac.  The behavior is the same.  What am I doing wrong?



---------------------------------------------------------------------
To unsubscribe, e-mail: id3v2-unsubscribe at id3.org
For additional commands, e-mail: id3v2-help at id3.org


From iontodirel at yahoo.co.uk  Sat Aug 27 06:18:27 2005
From: iontodirel at yahoo.co.uk (Ion Todirel)
Date: Sat, 27 Aug 2005 14:18:27 +0100 (BST)
Subject: [ID3 Dev] Syncsafe integers and 2.3/2.4
In-Reply-To: <20050827033626.GA22938@ayup.limey.net>
Message-ID: <20050827131827.34767.qmail@web86902.mail.ukl.yahoo.com>

If you wont to understand how something works you mast broks it in parts. Play with Mirosoft Windows Media Player SDK 10 or with Windows Media Format 9.5 SDK (in .NET), check winamp, way iTunes is so important for you? its more important to support standards! When in some case something doesn't work its best to not do it, in this case heuristical algoritm, in some cases it will not work. Like i say, way iTunes?

Ben Bennett <fiji at ayup.limey.net> wrote:On Mon, Aug 22, 2005 at 03:00:37PM +0100, Ion Todirel wrote:
> First you cannot determine if is syncsafe or not! based on MSB in
> all bytes. Second in your library you only can put an option to read
> frame size as syncsafe or as normal integers, you cannot read

Unfortunately that won't do it since iTunes leaves frames alone if it
hasn't changed them, so you may have a mix.

I have a heuristic in my parser that looks at the unsynchronized frame
size and check to see if there is a legal frame identifier following
it or if we have hit padding or the end of the file. I also check the
size of the following frame to make sure it is less than the tag
size. If either the frame id or the size are invalid I sanity check
the original size to see if it leads to a valid frame. If that check
fails too, I give up and go with the unsynchronized size.

It is clearly not perfect, but this seems to do a decent job of
catching the errors.

> ID3v2.3 or ID3v2.4 as normal because iTunes write these information
> as normal integers, look how WindowsMedia Player or Winamp write
> these, if what you say is true then iTunes does'n support ID3v2
> standard.

I agree. I have checked a few tools and some follow the spec... sadly
iTunes doesn't.


Anyway, you can get the tag I was playing with from:
http://limey.net/~fiji/t/id3v2.4_itunes-4.9.0.17.mp3
As the name suggests, that is an ID3v2.4 tag written by the latest
version of iTunes for Windows: 4.9.0.17.

The COMM frame is where the issue arises in that frame since it is the
only one long enough to trigger the issue. But it happens with any
frame larger than 127 bytes.

The frame in question looks like:
Header (10 Bytes):
43 4f 4d 4d Frame ID $xx xx xx xx (four characters)
00 00 01 05 Size 4 * %0xxxxxxx
00 00 Flags $xx xx

Payload (261 bytes):
00 Text encoding $xx
65 6e 67 Language $xx $xx $xx
00 Short content $00 (00)
(256 bytes) The actual text 

This frame is followed by padding out to the end of the tag.


The Frame ID is "COMM", and there are no special flags. I will get to
the size in a minute. The text encoding is $00 which means ISO-8859-1
(Latin1). There is no short content so you just see $00 indicating
the end of the string (since ISO-8859-1 is not a wide char there is no
second $00). Finally the actual text is 1234567890 repeated 25 times,
followed by 12FOO for a total of 255 characters + 1 byte for the null
at the end.

So Looking more closely at the size, the hex values:
00 00 01 05
Are the bit strings:
00000000 00000000 00000001 00000101
Which if syncsafe would drop the most significant bit from each byte:
0000 00000000 00000000 10000101
Which when converted to base-10 is 134.

However if we don't unsynchronize the number it is 261. Which matches
the actual frame payload size.

Hopefully this proves to you that iTunes is totally borked in the
2.4 frame length handling.

-ben

---------------------------------------------------------------------
To unsubscribe, e-mail: id3v2-unsubscribe at id3.org
For additional commands, e-mail: id3v2-help at id3.org



		
---------------------------------
How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos. Get Yahoo! Photos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.id3.org/pipermail/id3v2/attachments/20050827/3c4cb53c/attachment.html>

From iontodirel at yahoo.co.uk  Mon Aug 22 06:50:39 2005
From: iontodirel at yahoo.co.uk (Ion Todirel)
Date: Mon, 22 Aug 2005 14:50:39 +0100 (BST)
Subject: [ID3 Dev] Padding
In-Reply-To: <e81081bd050822000013f793c1@mail.gmail.com>
Message-ID: <20050822135039.91270.qmail@web86910.mail.ukl.yahoo.com>

Informal standard                                             M. Nilsson
Document: id3v2.4.0-structure.txt                      1st November 2000

ID3 tag version 2.4.0 - Main Structure
 
--------------------------------------------------------------------------------------------------------

3.3.   Padding (http://www.id3.org/id3v2.4.0-structure.txt)

   It is OPTIONAL to include padding after the final frame (at the end of the ID3 tag), making the size of all the frames together smaller  than the size given in the tag header. Tag size included on tag header, dont include padding? 

   A possible purpose of this padding is to allow for adding a few additional frames or enlarge existing frames within the tag without having to rewrite the entire file. If padding is used to add aditional frames, where is the frame ID since ID is four null chars? or if padding is used to enlarge existing frames how to know what frame are enlarged? nothing is said on id3.org! where to get this informations? please help!

And if footer is writting to end of file, is'n possible to write an ID3v1 Tag, right? since ID3v1 tag is located from 0 to 128 bytes at end of file.

Thanks.

Pyt <py.thoulon at gmail.com> wrote:
Padding is the bunch of null chars that lie between the end of the last frame and the end of the tag (the tag size, as given in the tag header, includes padding). A null char where you would expect a frame ID indicates start of padding (unless you're past the end of the tag). 
 
Rgds,
Pyt.

 
On 8/20/05, Ion Todirel <iontodirel at yahoo.co.uk> wrote: 

Ion Todirel <iontodirel at yahoo.co.uk> wrote: Please help me, how to detect padding? four zeroed chars?


---------------------------------
To help you stay safe and secure online, we've developed the all new Yahoo! Security Centre. 







---------------------------------
How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos. Get Yahoo! Photos 






		
---------------------------------
Yahoo! Messenger  NEW - crystal clear PC to PC calling worldwide with voicemail 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.id3.org/pipermail/id3v2/attachments/20050822/4bb03a5d/attachment.html>

From iontodirel at yahoo.co.uk  Sun Aug 21 21:40:51 2005
From: iontodirel at yahoo.co.uk (Ion Todirel)
Date: Mon, 22 Aug 2005 05:40:51 +0100 (BST)
Subject: [ID3 Dev] Syncsafe integers and 2.3/2.4
In-Reply-To: <20050821173852.n8nisjr5q7pcsk88@webmail.mit.edu>
Message-ID: <20050822044051.70259.qmail@web86910.mail.ukl.yahoo.com>

To Ben Bennett:
 
Both 2.3 and 2.4 use Syncsafe integers. ID3v2.4 its a nice one, i recommend you to implement yourself ID3v2.4/Reader-Writer and threat there your syncsafe problems, also if you you have problems with syncsafe email me, i develop some nice methods.


Robert Crowell <crowell at MIT.EDU> wrote:I am working on a python library to read (and possibly write) id3v2 tags.

Reading over the specification several times, I wasn't able to determine whether
or not id3v2.3 frames use the 'synchsafe integers' that 2.4 frames do. (I
understand that both 2.3 and 2.4 use synchsafe ints for the _tag_ size, but I
couldn't determine what the _frame_ behavior is.)

While the specification seems to indicate that v2.3 frames DO NOT use synchsafe
ints and v2.4 frames DO use synchsafe integers, I am encountering some odd
behavior. Using the 'convert ID3 tags' feature in iTunes 4.9, I convert a v2.3
tag to v2.4, but the frame size bytes don't seem to be any different in the 2.3
and 2.4 versions. There is a long USER tag in one of my mp3 files (that was
created by an unknown application) that contains 370 ASCII characters.

The 4 size bytes in this frame header seem to be encoding this value as a
NON-synchsafe integer. In my code, when 2.3 is detected it expects
non-synchsafe integers for frame sizes and everything is fine. However, after
converting this tag to 2.4 in iTunes, it interprets the size as a synchsafe
integer, and because iTunes did not modify the size bytes when converting from
2.3 to 2.4, it gets the frame size wrong. It decides the frame is only 242
bytes long rather than the correct 370. (size:0,0,1,114 -> 370 normal, 242
synchsafe)

This causes a huge problem because not only is the USER tag not read completely,
but my code mistakenly interprets the next 10 bytes as a new frame header. 
Because the size portion of this header is clearly invalid (since this isn't a
real header at all), my code believes the size of the next frame is huge,
interpreting 'manc' as the frame size bytes.

size bytes: $00 00 01 72 (for both 2.3 and 2.4)
2.3 size: 370 (correct, assuming non-synchsafe int)
2.4 size: 242 (incorrect, assuming synchsafe int)

Is iTunes at fault for not converting the USER frame's size to a synchsafe
integer when converting a tag to 2.4? Am I mis-reading the ID3v2
specification? If it turns out that iTunes is doing something silly, how would
I detect it in my code?

Thanks for your help.

-Rob Crowell

P.S. I have also tested the tags generated by the Media Rage application for
Mac. The behavior is the same. What am I doing wrong?



---------------------------------------------------------------------
To unsubscribe, e-mail: id3v2-unsubscribe at id3.org
For additional commands, e-mail: id3v2-help at id3.org



		
---------------------------------
To help you stay safe and secure online, we've developed the all new Yahoo! Security Centre.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.id3.org/pipermail/id3v2/attachments/20050822/21a91d9f/attachment.html>

From iontodirel at yahoo.co.uk  Sat Aug 27 06:23:58 2005
From: iontodirel at yahoo.co.uk (Ion Todirel)
Date: Sat, 27 Aug 2005 14:23:58 +0100 (BST)
Subject: [ID3 Dev] Syncsafe integers and 2.3/2.4
In-Reply-To: <20050827033626.GA22938@ayup.limey.net>
Message-ID: <20050827132358.90062.qmail@web86908.mail.ukl.yahoo.com>

You have some implementetion of a language (ISO-639-2) class/struct/enum? to convert in and from.
		
---------------------------------
To help you stay safe and secure online, we've developed the all new Yahoo! Security Centre.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.id3.org/pipermail/id3v2/attachments/20050827/0323d2f7/attachment.html>

From fiji at ayup.limey.net  Fri Aug 26 20:36:26 2005
From: fiji at ayup.limey.net (Ben Bennett)
Date: Fri, 26 Aug 2005 23:36:26 -0400
Subject: [ID3 Dev] Syncsafe integers and 2.3/2.4
In-Reply-To: <20050822140037.95547.qmail@web86902.mail.ukl.yahoo.com>
References: <20050822134016.GA12114@ayup.limey.net> <20050822140037.95547.qmail@web86902.mail.ukl.yahoo.com>
Message-ID: <20050827033626.GA22938@ayup.limey.net>

On Mon, Aug 22, 2005 at 03:00:37PM +0100, Ion Todirel wrote:
> First you cannot determine if is syncsafe or not! based on MSB in
> all bytes. Second in your library you only can put an option to read
> frame size as syncsafe or as normal integers, you cannot read

Unfortunately that won't do it since iTunes leaves frames alone if it
hasn't changed them, so you may have a mix.

I have a heuristic in my parser that looks at the unsynchronized frame
size and check to see if there is a legal frame identifier following
it or if we have hit padding or the end of the file.  I also check the
size of the following frame to make sure it is less than the tag
size.  If either the frame id or the size are invalid I sanity check
the original size to see if it leads to a valid frame.  If that check
fails too, I give up and go with the unsynchronized size.

It is clearly not perfect, but this seems to do a decent job of
catching the errors.

> ID3v2.3 or ID3v2.4 as normal because iTunes write these information
> as normal integers, look how WindowsMedia Player or Winamp write
> these, if what you say is true then iTunes does'n support ID3v2
> standard.

I agree.  I have checked a few tools and some follow the spec... sadly
iTunes doesn't.


Anyway, you can get the tag I was playing with from:
	http://limey.net/~fiji/t/id3v2.4_itunes-4.9.0.17.mp3
As the name suggests, that is an ID3v2.4 tag written by the latest
version of iTunes for Windows: 4.9.0.17.

The COMM frame is where the issue arises in that frame since it is the
only one long enough to trigger the issue.  But it happens with any
frame larger than 127 bytes.

The frame in question looks like:
 Header (10 Bytes):
   43 4f 4d 4d    Frame ID        $xx xx xx xx  (four characters)
   00 00 01 05    Size        4 * %0xxxxxxx
   00 00          Flags           $xx xx
 
 Payload (261 bytes):
   00		  Text encoding   $xx
   65 6e 67       Language        $xx $xx $xx
   00             Short content   <encoded text string> $00 (00)
   (256 bytes)    The actual text <full encoded text string>

This frame is followed by padding out to the end of the tag.


The Frame ID is "COMM", and there are no special flags.  I will get to
the size in a minute.  The text encoding is $00 which means ISO-8859-1
(Latin1).  There is no short content so you just see $00 indicating
the end of the string (since ISO-8859-1 is not a wide char there is no
second $00).  Finally the actual text is 1234567890 repeated 25 times,
followed by 12FOO for a total of 255 characters + 1 byte for the null
at the end.

So Looking more closely at the size, the hex values:
  00 00 01 05
Are the bit strings:
  00000000 00000000 00000001 00000101
Which if syncsafe would drop the most significant bit from each byte:
      0000 00000000 00000000 10000101
Which when converted to base-10 is 134.

However if we don't unsynchronize the number it is 261.  Which matches
the actual frame payload size.

Hopefully this proves to you that iTunes is totally borked in the
2.4 frame length handling.

		-ben

---------------------------------------------------------------------
To unsubscribe, e-mail: id3v2-unsubscribe at id3.org
For additional commands, e-mail: id3v2-help at id3.org



From iontodirel at yahoo.co.uk  Tue Aug 23 15:12:42 2005
From: iontodirel at yahoo.co.uk (Ion Todirel)
Date: Tue, 23 Aug 2005 23:12:42 +0100 (BST)
Subject: [ID3 Dev] Padding
In-Reply-To: <20050823211031.GA26928@ayup.limey.net>
Message-ID: <20050823221243.41855.qmail@web86908.mail.ukl.yahoo.com>

thanks very much Ben, i stiil dont understand why footer and padding are mutually exclusive? if i have this situation, all sems to be OK?
 
<tag header>
<TIT1 header><TIT1 contents>
<TIT2 header><TIT2 contents>
<padding>
<mp3 audio data>
<tag footer>
<ID3v1 tag>
 
and second problem:
 
here is a simple tag:
 
<tag header>
<"frame ID" header><"frame ID" content>
...
<"frame IDi" header><"frame IDi" content>
...
<"frame IDn" header><"frame IDn" content> (last frame (normal frame))

<padding>                                                 (lets say 100%)
<audio data>
<ID3v1>







 
Ben, way just not add two new frames like this?:
 
<tag header>
<"frame ID" header><"frame ID" content>
...
<"frame IDi" header><"frame IDi" content>
...
<"frame IDn" header><"frame IDn" content>

<"frame IDn+1" content><"frame IDn+1" content> (added frame)

<"frame IDn+2" content><"frame IDn+2" content> (added frame)                                 
<padding>                                                          (after adding two frames padding = padding - x bytes)   
<audio data>
<ID3v1>







 
but add like you say?:
 
<tag header>
<"frame ID1" header>
<"frame ID2" header>
<"frame ID3" header><"frame ID3" content>
...
<"frame IDi" header><"frame IDi" content>
...
<"frame IDn" header><"frame IDn" content> (last frame (normal frame))

<"frame ID1" content>          
<"frame ID2" content>                                  
<padding>                                                 (same situation) 
<audio data>
<ID3v1>
 
Thanks, again









Ben Bennett <fiji at ayup.limey.net> wrote:On Tue, Aug 23, 2005 at 10:00:26PM +0100, Ion Todirel wrote:
> If i use padding to add a new frame: let's say that i have a tag with TIT1 and TIT2: 
> 
> 
> 
> 
> 
> 
> 
> 

Yes.

> 
> if i add a new frame (TALB) using padding tag will look like this?:
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 

No.

The TALB is a frame and MUST be inside the ID3 tag. The padding also
falls inside the tag, (i.e. is included in the tag size) but MUST
follow all frames. So it will look like:













> if frame TALB already exists in the tag:
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 

Okay.

> and i extend it using padding:
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 

Again, no. You can not split the additional content from TALB. There
can only be one TALB frame per tag. So it will look like:













> But how to detect when padding starts? just before last frame, but how to detect when is last frame?

Padding falls after all frames. It is the last thing in the tag. So
if you expect to see a frame header but read all nuls then you are
into the padding.

The only point to padding is to prevent rewriting all of the content
in the file if a tag grows. So if you have N bytes of padding at the
end of a frame and you grow the content by N bytes or less, you can
just rewrite the tag and shrink the padding. But if you need to grow
something by more than N bytes then you need to fiddle with the
remainder of the file as well.

Does that help?

-ben

---------------------------------------------------------------------
To unsubscribe, e-mail: id3v2-unsubscribe at id3.org
For additional commands, e-mail: id3v2-help at id3.org



		
---------------------------------
How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos. Get Yahoo! Photos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.id3.org/pipermail/id3v2/attachments/20050823/e9b54ac3/attachment.html>

From richard.pfaller at web.de  Thu Aug  4 19:25:08 2005
From: richard.pfaller at web.de (Richard Pfaller)
Date: Fri, 05 Aug 2005 04:25:08 +0200
Subject: [ID3 Dev] Question on ID3v2.3.0 ExtendedHeaderSize
Message-ID: <42F2CE04.20907@web.de>

Helo,

I would like to ask if the ExtendedHeaderSize of v2.3.0 is safed as an 
32bit Syncsafe Integer like in v2.4.0, and if it is not so in what way a 
standard conform prog should safe and read it.

-rmp

---------------------------------------------------------------------
To unsubscribe, e-mail: id3v2-unsubscribe at id3.org
For additional commands, e-mail: id3v2-help at id3.org